# Hypernetworks for Continual Semi-Supervised Learning

arXiv: https://arxiv.org/abs/2110.01856 · HTML: https://mlsanity.com/paper/2110.01856/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 27 references that are in the map.
- First posted on arXiv 5 Oct 2021.

## Builds on (most cited first, 18 of 18)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Gradient based sample selection for online continual learning](https://mlsanity.com/paper/1903.08671/) (2019)
- [Conditional Image Synthesis With Auxiliary Classifier GANs](https://mlsanity.com/paper/1610.09585/) (2016)
- [Variational Continual Learning](https://mlsanity.com/paper/1710.10628/) (2017)
- [Continual learning with hypernetworks](https://mlsanity.com/paper/1906.00695/) (2019)
- [Continual Unsupervised Representation Learning](https://mlsanity.com/paper/1910.14481/) (2019)
- [A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning](https://mlsanity.com/paper/2001.00689/) (2020)
- [Meta-Consolidation for Continual Learning](https://mlsanity.com/paper/2010.00352/) (2020)
- [Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer](https://mlsanity.com/paper/2101.09536/) (2021)

## Built on by (most cited first, 11 of 11)

- [A Probabilistic Framework for Lifelong Test-Time Adaptation](https://mlsanity.com/paper/2212.09713/) (2022)
- [Label Shift Adapter for Test-Time Adaptation under Covariate and Label Shifts](https://mlsanity.com/paper/2308.08810/) (2023)
- [HackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learning](https://mlsanity.com/paper/2406.03997/) (2024)
- [HMOE: Hypernetwork-based Mixture of Experts for Domain Generalization](https://mlsanity.com/paper/2211.08253/) (2022)
- [Learn from Yesterday: A Semi-Supervised Continual Learning Method for Supervision-Limited Text-to-SQL Task Streams](https://mlsanity.com/paper/2211.11226/) (2022)
- [Scalable and Efficient Continual Learning from Demonstration via a Hypernetwork-generated Stable Dynamics Model](https://mlsanity.com/paper/2311.03600/) (2023)
- [Transferring Unconditional to Conditional GANs with Hyper-Modulation](https://mlsanity.com/paper/2112.02219/) (2021)
- [Contrastive Learning for Online Semi-Supervised General Continual Learning](https://mlsanity.com/paper/2207.05615/) (2022)
- [Towards Label-Efficient Incremental Learning: A Survey](https://mlsanity.com/paper/2302.00353/) (2023)
- [Semi-Supervised Lifelong Language Learning](https://mlsanity.com/paper/2211.13050/) (2022)
- [CLA: Latent Alignment for Online Continual Self-Supervised Learning](https://mlsanity.com/paper/2507.10434/) (2025)

## Abstract (arXiv)

Learning from data sequentially arriving, possibly in a non i.i.d. way, with changing task distribution over time is called continual learning. Much of the work thus far in continual learning focuses on supervised learning and some recent works on unsupervised learning. In many domains, each task contains a mix of labelled (typically very few) and unlabelled (typically plenty) training examples, which necessitates a semi-supervised learning approach. To address this in a continual learning setting, we propose a framework for semi-supervised continual learning called Meta-Consolidation for Continual Semi-Supervised Learning (MCSSL). Our framework has a hypernetwork that learns the meta-distribution that generates the weights of a semi-supervised auxiliary classifier generative adversarial network (textitSemi-ACGAN) as the base network. We consolidate the knowledge of sequential tasks in the hypernetwork, and the base network learns the semi-supervised learning task. Further, we present textitSemi-Split CIFAR-10, a new benchmark for continual semi-supervised learning, obtained by modifying the textitSplit CIFAR-10 dataset, in which the tasks with labelled and unlabelled data arrive sequentially. Our proposed model yields significant improvements in the continual semi-supervised learning setting. We compare the performance of several existing continual learning approaches on the proposed continual semi-supervised learning benchmark of the Semi-Split CIFAR-10 dataset.