# Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer

arXiv: https://arxiv.org/abs/2101.09536 · HTML: https://mlsanity.com/paper/2101.09536/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 63 references that are in the map.
- First posted on arXiv 23 Jan 2021.

## Builds on (most cited first, 30 of 50)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Gradient based sample selection for online continual learning](https://mlsanity.com/paper/1903.08671/) (2019)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data](https://mlsanity.com/paper/2002.11297/) (2020)
- [Online Continual Learning with Maximally Interfered Retrieval](https://mlsanity.com/paper/1908.04742/) (2019)
- [Measuring Catastrophic Forgetting in Neural Networks](https://mlsanity.com/paper/1708.02072/) (2017)
- [On Tiny Episodic Memories in Continual Learning](https://mlsanity.com/paper/1902.10486/) (2019)
- [CORe50: a New Dataset and Benchmark for Continuous Object Recognition](https://mlsanity.com/paper/1705.03550/) (2017)
- [Continual learning with hypernetworks](https://mlsanity.com/paper/1906.00695/) (2019)
- [Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines](https://mlsanity.com/paper/1810.12488/) (2018)
- [Meta-Learning Representations for Continual Learning](https://mlsanity.com/paper/1905.12588/) (2019)

## Built on by (most cited first, 30 of 30)

- [DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning](https://mlsanity.com/paper/2204.04799/) (2022)
- [Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning](https://mlsanity.com/paper/2106.09701/) (2021)
- [The CLEAR Benchmark: Continual LEArning on Real-World Imagery](https://mlsanity.com/paper/2201.06289/) (2022)
- [A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning](https://mlsanity.com/paper/2307.09218/) (2023)
- [Recent Advances of Continual Learning in Computer Vision: An Overview](https://mlsanity.com/paper/2109.11369/) (2021)
- [Transfer without Forgetting](https://mlsanity.com/paper/2206.00388/) (2022)
- [Continual Sequence Generation with Adaptive Compositional Modules](https://mlsanity.com/paper/2203.10652/) (2022)
- [A Multi-Head Model for Continual Learning via Out-of-Distribution Replay](https://mlsanity.com/paper/2208.09734/) (2022)
- [AIDE: An Automatic Data Engine for Object Detection in Autonomous Driving](https://mlsanity.com/paper/2403.17373/) (2024)
- [A soft nearest-neighbor framework for continual semi-supervised learning](https://mlsanity.com/paper/2212.05102/) (2022)
- [Label-Efficient Online Continual Object Detection in Streaming Video](https://mlsanity.com/paper/2206.00309/) (2022)
- [NovGrid: A Flexible Grid World for Evaluating Agent Response to Novelty](https://mlsanity.com/paper/2203.12117/) (2022)
- [Meta-Learning with Less Forgetting on Large-Scale Non-Stationary Task Distributions](https://mlsanity.com/paper/2209.01501/) (2022)
- [Hypernetworks for Continual Semi-Supervised Learning](https://mlsanity.com/paper/2110.01856/) (2021)
- [Beyond Supervised Continual Learning: a Review](https://mlsanity.com/paper/2208.14307/) (2022)
- [Teamwork Is Not Always Good: An Empirical Study of Classifier Drift in Class-incremental Information Extraction](https://mlsanity.com/paper/2305.16559/) (2023)
- [Dynamic Sub-graph Distillation for Robust Semi-supervised Continual Learning](https://mlsanity.com/paper/2312.16409/) (2023)
- [Exploring Covariate and Concept Shift for Detection and Calibration of Out-of-Distribution Data](https://mlsanity.com/paper/2110.15231/) (2021)
- [Learn from Yesterday: A Semi-Supervised Continual Learning Method for Supervision-Limited Text-to-SQL Task Streams](https://mlsanity.com/paper/2211.11226/) (2022)
- [Self-Reinforcing Prototype Evolution with Dual-Knowledge Cooperation for Semi-Supervised Lifelong Person Re-Identification](https://mlsanity.com/paper/2507.01884/) (2025)
- [Contrastive Learning for Online Semi-Supervised General Continual Learning](https://mlsanity.com/paper/2207.05615/) (2022)
- [Divide-and-Conquer for Enhancing Unlabeled Learning, Stability, and Plasticity in Semi-supervised Continual Learning](https://mlsanity.com/paper/2508.05316/) (2025)
- [Towards Label-Efficient Incremental Learning: A Survey](https://mlsanity.com/paper/2302.00353/) (2023)
- [Dual-CBA: Improving Online Continual Learning via Dual Continual Bias Adaptors from a Bi-level Optimization Perspective](https://mlsanity.com/paper/2408.13991/) (2024)
- [Semi-Supervised Lifelong Language Learning](https://mlsanity.com/paper/2211.13050/) (2022)
- [Continual Learning for Multimodal Data Fusion of a Soft Gripper](https://mlsanity.com/paper/2409.13792/) (2024)
- [CLA: Latent Alignment for Online Continual Self-Supervised Learning](https://mlsanity.com/paper/2507.10434/) (2025)
- [Rehearsal-free and Task-free Online Continual Learning With Contrastive Prompt](https://mlsanity.com/paper/2510.00467/) (2025)
- [Audio-Visual Continual Test-Time Adaptation without Forgetting](https://mlsanity.com/paper/2602.18528/) (2026)
- [A Distinct Unsupervised Reference Model From The Environment Helps Continual Learning](https://mlsanity.com/paper/2301.04506/) (2023)

## Abstract (arXiv)

Rehearsal is a critical component for class-incremental continual learning, yet it requires a substantial memory budget. Our work investigates whether we can significantly reduce this memory budget by leveraging unlabeled data from an agent's environment in a realistic and challenging continual learning paradigm. Specifically, we explore and formalize a novel semi-supervised continual learning (SSCL) setting, where labeled data is scarce yet non-i.i.d. unlabeled data from the agent's environment is plentiful. Importantly, data distributions in the SSCL setting are realistic and therefore reflect object class correlations between, and among, the labeled and unlabeled data distributions. We show that a strategy built on pseudo-labeling, consistency regularization, Out-of-Distribution (OoD) detection, and knowledge distillation reduces forgetting in this setting. Our approach, DistillMatch, increases performance over the state-of-the-art by no less than 8.7% average task accuracy and up to 54.5% average task accuracy in SSCL CIFAR-100 experiments. Moreover, we demonstrate that DistillMatch can save up to 0.23 stored images per processed unlabeled image compared to the next best method which only saves 0.08. Our results suggest that focusing on realistic correlated distributions is a significantly new perspective, which accentuates the importance of leveraging the world's structure as a continual learning strategy.