mlsanityData to 7 Oct 2026

Masked Autoencoders Are Scalable Vision Learners

arXiv 2111.06377 ↗First posted 11 Nov 2021cs.CVcited by 7,339

Not on the map. Why?

Cited by 363 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper shows that masked autoencoders (MAE) are scalable self-supervised learners for computer vision. Our MAE approach is simple: we mask random patches of the input image and reconstruct the missing pixels. It is based on two core designs. First, we develop an asymmetric encoder-decoder architecture, with an encoder that operates only on the visible subset of patches (without mask tokens), along with a lightweight decoder that reconstructs the original image from the latent representation and mask tokens. Second, we find that masking a high proportion of the input image, e.g., 75%, yields a nontrivial and meaningful self-supervisory task. Coupling these two designs enables us to train large models efficiently and effectively: we accelerate training (by 3x or more) and improve accuracy. Our scalable approach allows for learning high-capacity models that generalize well: e.g., a vanilla ViT-Huge model achieves the best accuracy (87.8%) among methods that use only ImageNet-1K data. Transfer performance in downstream tasks outperforms supervised pre-training and shows promising scaling behavior.

Key facts. Cited by 7,339 papers in the map (data through 7 Oct 2026). Builds on 52 of its 73 references that are in the map. First posted on arXiv 11 Nov 2021.

Builds on

52 of 73 references.

References · 52

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  8. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  9. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  10. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  11. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  12. 2017Mask R-CNNcited by 7,259
38 more
  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  3. 2016Layer Normalizationcited by 5,579
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2017Neural Discrete Representation Learningcited by 4,650
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  13. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  14. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  15. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  16. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  17. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  18. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  19. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  20. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  21. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  22. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  23. 2019Natural Adversarial Examplescited by 1,401
  24. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  25. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  26. 2014How transferable are features in deep neural networks?cited by 1,216
  27. 2016Colorful Image Colorizationcited by 1,113
  28. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  29. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  30. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  31. 2016Deep Networks with Stochastic Depthcited by 973
  32. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  33. 2017Large Batch Training of Convolutional Networkscited by 529
  34. 2021Early Convolutions Help Transformers See Bettercited by 395
  35. 2019Fixing the train-test resolution discrepancycited by 209
  36. 2021Self-supervised Pretraining of Visual Features in the Wildcited by 197
  37. 2021VOLO: Vision Outlooker for Visual Recognitioncited by 164
  38. 2016Learning Features by Watching Objects Movecited by 138

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 31; 2022: 830; 2023: 1679; 2024: 1784; 2025: 1678; 2026: 1337.

7,339 papers.

Sort:

By area:

  1. 2023Segment Anythingcited by 8,158
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2022A ConvNet for the 2020scited by 3,545
  4. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  5. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  6. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  7. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  8. 2022Visual Prompt Tuningcited by 1,674
  9. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  10. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  11. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  12. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
38 more
  1. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  2. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  3. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  4. 2022Token Merging: Your ViT But Fastercited by 995
  5. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  6. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  7. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  8. 2023Vision Transformers Need Registerscited by 844
  9. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  10. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  11. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  12. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  13. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  14. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  15. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  16. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  17. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  18. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  19. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  20. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  21. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  22. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  23. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  24. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  25. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
  26. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  27. 2022Vision Transformer Adapter for Dense Predictionscited by 564
  28. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 536
  29. 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 530
  30. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  31. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
  32. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  33. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 495
  34. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
  35. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  36. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  37. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  38. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467

50 most cited of 7,339 All 7,339 (JSON)