# Incremental Classifier Learning with Generative Adversarial Networks

arXiv: https://arxiv.org/abs/1802.00853 · HTML: https://mlsanity.com/paper/1802.00853/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Feb 2018.

## Built on by (most cited first, 30 of 32)

- [Online Continual Learning in Image Classification: An Empirical Survey](https://mlsanity.com/paper/2101.10423/) (2021)
- [Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations](https://mlsanity.com/paper/2103.06342/) (2021)
- [RECALL: Replay-based Continual Learning in Semantic Segmentation](https://mlsanity.com/paper/2108.03673/) (2021)
- [Recent Advances of Continual Learning in Computer Vision: An Overview](https://mlsanity.com/paper/2109.11369/) (2021)
- [A Data-Free Approach to Mitigate Catastrophic Forgetting in Federated Class Incremental Learning for Vision Tasks](https://mlsanity.com/paper/2311.07784/) (2023)
- [EEC: Learning to Encode and Regenerate Images for Continual Learning](https://mlsanity.com/paper/2101.04904/) (2021)
- [Robust and Resource-Efficient Data-Free Knowledge Distillation by Generative Pseudo Replay](https://mlsanity.com/paper/2201.03019/) (2022)
- [Attribution-aware Weight Transfer: A Warm-Start Initialization for Class-Incremental Semantic Segmentation](https://mlsanity.com/paper/2210.07207/) (2022)
- [Continual Learning in Deep Networks: an Analysis of the Last Layer](https://mlsanity.com/paper/2106.01834/) (2021)
- [Text-Enhanced Data-free Approach for Federated Class-Incremental Learning](https://mlsanity.com/paper/2403.14101/) (2024)
- [Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting](https://mlsanity.com/paper/2502.02797/) (2025)
- [Towards Unsupervised Open World Semantic Segmentation](https://mlsanity.com/paper/2201.01073/) (2022)
- [Do Not Forget to Attend to Uncertainty while Mitigating Catastrophic Forgetting](https://mlsanity.com/paper/2102.01906/) (2021)
- [When Meta-Learning Meets Online and Continual Learning: A Survey](https://mlsanity.com/paper/2311.05241/) (2023)
- [Theoretical Understanding of the Information Flow on Continual Learning Performance](https://mlsanity.com/paper/2204.12010/) (2022)
- [Evidential Deep Learning for Class-Incremental Semantic Segmentation](https://mlsanity.com/paper/2212.02863/) (2022)
- [Context-Free Synthetic Data Mitigates Forgetting](https://mlsanity.com/paper/2505.13811/) (2025)
- [Detecting and Learning the Unknown in Semantic Segmentation](https://mlsanity.com/paper/2202.08700/) (2022)
- [RECALL+: Adversarial Web-based Replay for Continual Learning in Semantic Segmentation](https://mlsanity.com/paper/2309.10479/) (2023)
- [Privacy-Preserving Synthetic Continual Semantic Segmentation for Robotic Surgery](https://mlsanity.com/paper/2402.05860/) (2024)
- [Energy Aligning for Biased Models](https://mlsanity.com/paper/2106.03343/) (2021)
- [Accretionary Learning with Deep Neural Networks](https://mlsanity.com/paper/2111.10857/) (2021)
- [SAM-kNN Regressor for Online Learning in Water Distribution Networks](https://mlsanity.com/paper/2204.01436/) (2022)
- [Investigating the Impact of Weight Sharing Decisions on Knowledge Transfer in Continual Learning](https://mlsanity.com/paper/2311.09506/) (2023)
- [Exemplar Masking for Multimodal Incremental Learning](https://mlsanity.com/paper/2412.09549/) (2024)
- [Progressive Voronoi Diagram Subdivision: Towards A Holistic Geometric Framework for Exemplar-free Class-Incremental Learning](https://mlsanity.com/paper/2207.14202/) (2022)
- [Dissecting Continual Learning a Structural and Data Analysis](https://mlsanity.com/paper/2301.01033/) (2023)
- [Detachedly Learn a Classifier for Class-Incremental Learning](https://mlsanity.com/paper/2302.11730/) (2023)
- [A Methodology-Oriented Study of Catastrophic Forgetting in Incremental Deep Neural Networks](https://mlsanity.com/paper/2405.08015/) (2024)
- [GPS: Distilling Compact Memories via Grid-based Patch Sampling for Efficient Online Class-Incremental Learning](https://mlsanity.com/paper/2504.10409/) (2025)

## Abstract (arXiv)

In this paper, we address the incremental classifier learning problem, which suffers from catastrophic forgetting. The main reason for catastrophic forgetting is that the past data are not available during learning. Typical approaches keep some exemplars for the past classes and use distillation regularization to retain the classification capability on the past classes and balance the past and new classes. However, there are four main problems with these approaches. First, the loss function is not efficient for classification. Second, there is unbalance problem between the past and new classes. Third, the size of pre-decided exemplars is usually limited and they might not be distinguishable from unseen new classes. Forth, the exemplars may not be allowed to be kept for a long time due to privacy regulations. To address these problems, we propose (a) a new loss function to combine the cross-entropy loss and distillation loss, (b) a simple way to estimate and remove the unbalance between the old and new classes , and (c) using Generative Adversarial Networks (GANs) to generate historical data and select representative exemplars during generation. We believe that the data generated by GANs have much less privacy issues than real images because GANs do not directly copy any real image patches. We evaluate the proposed method on CIFAR-100, Flower-102, and MS-Celeb-1M-Base datasets and extensive experiments demonstrate the effectiveness of our method.