| | |
| | |
Stat |
Members: 3667 Articles: 2'599'751 Articles rated: 2609
07 February 2025 |
|
| | | |
|
Article overview
| |
|
Semi-MAE: Masked Autoencoders for Semi-supervised Vision Transformers | Haojie Yu
; Kang Zhao
; Xiaoming Xu
; | Date: |
4 Jan 2023 | Abstract: | Vision Transformer (ViT) suffers from data scarcity in semi-supervised
learning (SSL). To alleviate this issue, inspired by masked autoencoder (MAE),
which is a data-efficient self-supervised learner, we propose Semi-MAE, a pure
ViT-based SSL framework consisting of a parallel MAE branch to assist the
visual representation learning and make the pseudo labels more accurate. The
MAE branch is designed as an asymmetric architecture consisting of a
lightweight decoder and a shared-weights encoder. We feed the weakly-augmented
unlabeled data with a high masking ratio to the MAE branch and reconstruct the
missing pixels. Semi-MAE achieves 75.9% top-1 accuracy on ImageNet with 10%
labels, surpassing prior state-of-the-art in semi-supervised image
classification. In addition, extensive experiments demonstrate that Semi-MAE
can be readily used for other ViT models and masked image modeling methods. | Source: | arXiv, 2301.01431 | Services: | Forum | Review | PDF | Favorites |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
|
| |
|
|
|