# Contrastive Learning for Online Semi-Supervised General Continual Learning

arXiv: https://arxiv.org/abs/2207.05615 · HTML: https://mlsanity.com/paper/2207.05615/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 27 references that are in the map.
- First posted on arXiv 12 Jul 2022.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [Online Continual Learning in Image Classification: An Empirical Survey](https://mlsanity.com/paper/2101.10423/) (2021)
- [Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines](https://mlsanity.com/paper/1810.12488/) (2018)
- [Continual Unsupervised Representation Learning](https://mlsanity.com/paper/1910.14481/) (2019)
- [Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learning](https://mlsanity.com/paper/2103.13885/) (2021)
- [Unsupervised Progressive Learning and the STAM Architecture](https://mlsanity.com/paper/1904.02021/) (2019)
- [Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer](https://mlsanity.com/paper/2101.09536/) (2021)
- [Continual Semi-Supervised Learning through Contrastive Interpolation Consistency](https://mlsanity.com/paper/2108.06552/) (2021)
- [Unsupervised Continual Learning Via Pseudo Labels](https://mlsanity.com/paper/2104.07164/) (2021)
- [Hypernetworks for Continual Semi-Supervised Learning](https://mlsanity.com/paper/2110.01856/) (2021)

## Built on by (most cited first, 4 of 4)

- [Learning Representations on the Unit Sphere: Investigating Angular Gaussian and von Mises-Fisher Distributions for Online Continual Learning](https://mlsanity.com/paper/2306.03364/) (2023)
- [Domain-Aware Augmentations for Unsupervised Online General Continual Learning](https://mlsanity.com/paper/2309.06896/) (2023)
- [New metrics for analyzing continual learners](https://mlsanity.com/paper/2309.00462/) (2023)
- [From Offline to Online Memory-Free and Task-Free Continual Learning via Fine-Grained Hypergradients](https://mlsanity.com/paper/2502.18762/) (2025)

## Abstract (arXiv)

We study Online Continual Learning with missing labels and propose SemiCon, a new contrastive loss designed for partly labeled data. We demonstrate its efficiency by devising a memory-based method trained on an unlabeled data stream, where every data added to memory is labeled using an oracle. Our approach outperforms existing semi-supervised methods when few labels are available, and obtain similar results to state-of-the-art supervised methods while using only 2.6% of labels on Split-CIFAR10 and 10% of labels on Split-CIFAR100.