mlsanityData to 14 Oct 2026

Masked Autoencoders are Efficient Class Incremental Learners

arXiv 2308.12510 ↗First posted 24 Aug 2023cs.CV cs.AI cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Class Incremental Learning (CIL) aims to sequentially learn new classes while avoiding catastrophic forgetting of previous knowledge. We propose to use Masked Autoencoders (MAEs) as efficient learners for CIL. MAEs were originally designed to learn useful representations through reconstructive unsupervised learning, and they can be easily integrated with a supervised loss for classification. Moreover, MAEs can reliably reconstruct original input images from randomly selected patches, which we use to store exemplars from past tasks more efficiently for CIL. We also propose a bilateral MAE framework to learn from image-level and embedding-level fusion, which produces better-quality reconstructed images and more stable representations. Our experiments confirm that our approach performs better than the state-of-the-art on CIFAR-100, ImageNet-Subset, and ImageNet-Full. The code is available at https://github.com/scok30/MAE-CIL .

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 36 of its 47 references that are in the map. First posted on arXiv 24 Aug 2023.

Builds on

36 of 47 references.

References · 36

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  5. 2016Learning without Forgettingcited by 2,496
  6. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  7. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  8. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  9. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  10. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  11. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  12. 2018Experience Replay for Continual Learningcited by 1,095
24 more
  1. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  2. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  3. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  4. 2019Large Scale Incremental Learningcited by 732
  5. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 521
  6. 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 449
  7. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 423
  8. 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 372
  9. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  10. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  11. 2018Selective Experience Replay for Lifelong Learningcited by 241
  12. 2020Mnemonics Training: Multi-Class Incremental Learning without Forgettingcited by 221
  13. 2018Reinforced Continual Learningcited by 186
  14. 2019Class-incremental Learning via Deep Model Consolidationcited by 173
  15. 2021Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learningcited by 131
  16. 2020Generative Feature Replay For Class-Incremental Learningcited by 109
  17. 2020A Comprehensive Study of Class Incremental Learning Algorithms for Visual Taskscited by 107
  18. 2020Revisiting Training Strategies and Generalization Performance in Deep Metric Learningcited by 100
  19. 2022Memory Replay with Data Compression for Continual Learningcited by 82
  20. 2019Lifelong GAN: Continual Learning for Conditional Image Generationcited by 78
  21. 2020Memory Efficient Class-Incremental Learning for Image Classificationcited by 50
  22. 2023Class-Incremental Exemplar Compression for Class-Incremental Learningcited by 50
  23. 2021DualNet: Continual Learning, Fast and Slowcited by 39
  24. 2022SupMAE: Supervised Masked Autoencoders Are Efficient Vision Learnerscited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 1; 2026: 3.

10 papers.

  1. 2024Masked Image Modeling: A Surveycited by 29
  2. 2025Language Guided Concept Bottleneck Models for Interpretable Continual Learningcited by 28
  3. 2023Masked Modeling for Self-supervised Representation Learning on Vision and Beyondcited by 13
  4. 2024Exploiting the Semantic Knowledge of Pre-trained Text-Encoders for Continual Learningcited by 6
  5. 2024Masked Pre-training Enables Universal Zero-shot Denoisercited by 3
  6. 2024Masked Autoencoders are Parameter-Efficient Federated Continual Learnerscited by 3
  7. 2024Class incremental learning with probability dampening and cascaded gated classifiercited by 1
  8. 2026Few-shot Class-Incremental Learning via Generative Co-Memory Regularizationcited by 1
  9. 2026DRDN: Decoupled Representation Dynamic Network for From-Scratch ViT Class-Incremental Learningcited by 0
  10. 2026Lifelong Representations: A Survey on Continual Self-Supervised Learning for Vision Modelscited by 0