# Class-Incremental Learning with Generative Classifiers

arXiv: https://arxiv.org/abs/2104.10093 · HTML: https://mlsanity.com/paper/2104.10093/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 62 references that are in the map.
- First posted on arXiv 20 Apr 2021.

## Builds on (most cited first, 30 of 43)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [Overcoming catastrophic forgetting with hard attention to the task](https://mlsanity.com/paper/1801.01423/) (2018)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [Class-incremental learning: survey and performance evaluation on image classification](https://mlsanity.com/paper/2010.15277/) (2020)
- [Online Continual Learning with Maximally Interfered Retrieval](https://mlsanity.com/paper/1908.04742/) (2019)
- [Variational Continual Learning](https://mlsanity.com/paper/1710.10628/) (2017)
- [A note on the evaluation of generative models](https://mlsanity.com/paper/1511.01844/) (2015)
- [CORe50: a New Dataset and Benchmark for Continuous Object Recognition](https://mlsanity.com/paper/1705.03550/) (2017)
- [Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines](https://mlsanity.com/paper/1810.12488/) (2018)
- [Task-Free Continual Learning](https://mlsanity.com/paper/1812.03596/) (2018)
- [REMIND Your Neural Network to Prevent Catastrophic Forgetting](https://mlsanity.com/paper/1910.02509/) (2019)
- [Towards Robust Evaluations of Continual Learning](https://mlsanity.com/paper/1805.09733/) (2018)
- [Continual Prototype Evolution: Learning Online from Non-Stationary Data Streams](https://mlsanity.com/paper/2009.00919/) (2020)
- [Continuous Learning in Single-Incremental-Task Scenarios](https://mlsanity.com/paper/1806.08568/) (2018)
- [Generative Feature Replay For Class-Incremental Learning](https://mlsanity.com/paper/2004.09199/) (2020)
- [A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks](https://mlsanity.com/paper/2011.01844/) (2020)
- [Continual Deep Learning by Functional Regularisation of Memorable Past](https://mlsanity.com/paper/2004.14070/) (2020)
- [Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization](https://mlsanity.com/paper/1802.01569/) (2018)
- [Towards the first adversarially robust neural network model on MNIST](https://mlsanity.com/paper/1805.09190/) (2018)
- [Generative replay with feedback connections as a general strategy for continual learning](https://mlsanity.com/paper/1809.10635/) (2018)

## Built on by (most cited first, 30 of 39)

- [S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learning](https://mlsanity.com/paper/2207.12819/) (2022)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [Continual Learning and Catastrophic Forgetting](https://mlsanity.com/paper/2403.05175/) (2024)
- [Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inference](https://mlsanity.com/paper/2110.10031/) (2021)
- [A Continual Deepfake Detection Benchmark: Dataset, Methods, and Essentials](https://mlsanity.com/paper/2205.05467/) (2022)
- [Divide and not forget: Ensemble of selectively trained experts in Continual Learning](https://mlsanity.com/paper/2401.10191/) (2024)
- [It's All In the Teacher: Zero-Shot Quantization Brought Closer to the Teacher](https://mlsanity.com/paper/2203.17008/) (2022)
- [Two Complementary Perspectives to Continual Learning: Ask Not Only What to Optimize, But Also How](https://mlsanity.com/paper/2311.04898/) (2023)
- [Prediction Error-based Classification for Class-Incremental Learning](https://mlsanity.com/paper/2305.18806/) (2023)
- [Random Representations Outperform Online Continually Learned Representations](https://mlsanity.com/paper/2402.08823/) (2024)
- [Alleviating Representational Shift for Continual Fine-tuning](https://mlsanity.com/paper/2204.10535/) (2022)
- [Low-rank Prompt Interaction for Continual Vision-Language Retrieval](https://mlsanity.com/paper/2501.14369/) (2025)
- [DualCP: Rehearsal-Free Domain-Incremental Learning via Dual-Level Concept Prototype](https://mlsanity.com/paper/2503.18042/) (2025)
- [On Sequential Bayesian Inference for Continual Learning](https://mlsanity.com/paper/2301.01828/) (2023)
- [Dendritic Self-Organizing Maps for Continual Learning](https://mlsanity.com/paper/2110.13611/) (2021)
- [Learning to Learn for Few-shot Continual Active Learning](https://mlsanity.com/paper/2311.03732/) (2023)
- [Online Continual Learning via Logit Adjusted Softmax](https://mlsanity.com/paper/2311.06460/) (2023)
- [The Gaussian Discriminant Variational Autoencoder (GdVAE): A Self-Explainable Model with Counterfactual Explanations](https://mlsanity.com/paper/2409.12952/) (2024)
- [HyperMask: Adaptive Hypernetwork-based Masks for Continual Learning](https://mlsanity.com/paper/2310.00113/) (2023)
- [Task Confusion and Catastrophic Forgetting in Class-Incremental Learning: A Mathematical Framework for Discriminative and Generative Modelings](https://mlsanity.com/paper/2410.20768/) (2024)
- [Autoencoder-Based Hybrid Replay for Class-Incremental Learning](https://mlsanity.com/paper/2505.05926/) (2025)
- [Unveiling the Tapestry: the Interplay of Generalization and Forgetting in Continual Learning](https://mlsanity.com/paper/2211.11174/) (2022)
- [Mitigating Catastrophic Forgetting in the Incremental Learning of Medical Images](https://mlsanity.com/paper/2504.20033/) (2025)
- [Flashbacks to Harmonize Stability and Plasticity in Continual Learning](https://mlsanity.com/paper/2506.00477/) (2025)
- [Your VAR Model is Secretly an Efficient and Explainable Generative Classifier](https://mlsanity.com/paper/2510.12060/) (2025)
- [Distance Based Image Classification: A solution to generative classification's conundrum?](https://mlsanity.com/paper/2210.01349/) (2022)
- [eTag: Class-Incremental Learning with Embedding Distillation and Task-Oriented Generation](https://mlsanity.com/paper/2304.10103/) (2023)
- [Drift to Remember](https://mlsanity.com/paper/2409.13997/) (2024)
- [The Bayesian Approach to Continual Learning: An Overview](https://mlsanity.com/paper/2507.08922/) (2025)
- [Task-Agnostic Federated Continual Learning via Replay-Free Gradient Projection](https://mlsanity.com/paper/2509.21606/) (2025)

## Abstract (arXiv)

Incrementally training deep neural networks to recognize new classes is a challenging problem. Most existing class-incremental learning methods store data or use generative replay, both of which have drawbacks, while 'rehearsal-free' alternatives such as parameter regularization or bias-correction methods do not consistently achieve high performance. Here, we put forward a new strategy for class-incremental learning: generative classification. Rather than directly learning the conditional distribution p(y|x), our proposal is to learn the joint distribution p(x,y), factorized as p(x|y)p(y), and to perform classification using Bayes' rule. As a proof-of-principle, here we implement this strategy by training a variational autoencoder for each class to be learned and by using importance sampling to estimate the likelihoods p(x|y). This simple approach performs very well on a diverse set of continual learning benchmarks, outperforming generative replay and other existing baselines that do not store data.