# Generative Adversarial Network Training is a Continual Learning Problem

arXiv: https://arxiv.org/abs/1811.11083 · HTML: https://mlsanity.com/paper/1811.11083/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Nov 2018.

## Built on by (most cited first, 13 of 13)

- [A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning](https://mlsanity.com/paper/2307.09218/) (2023)
- [Efficient Feature Transformations for Discriminative and Generative Continual Learning](https://mlsanity.com/paper/2103.13558/) (2021)
- [StyleAlign: Analysis and Applications of Aligned StyleGAN Models](https://mlsanity.com/paper/2110.11323/) (2021)
- [Momentum Adversarial Distillation: Handling Large Distribution Shifts in Data-Free Knowledge Distillation](https://mlsanity.com/paper/2209.10359/) (2022)
- [IB-DRR: Incremental Learning with Information-Back Discrete Representation Replay](https://mlsanity.com/paper/2104.10588/) (2021)
- [Escaping the Verifier: Learning to Reason via Demonstrations](https://mlsanity.com/paper/2511.21667/) (2025)
- [Mind the Gap Between Synthetic and Real: Utilizing Transfer Learning to Probe the Boundaries of Stable Diffusion Generated Data](https://mlsanity.com/paper/2405.03243/) (2024)
- [Overcoming Mode Collapse with Adaptive Multi Adversarial Training](https://mlsanity.com/paper/2112.14406/) (2021)
- [Meta-Learned Attribute Self-Gating for Continual Generalized Zero-Shot Learning](https://mlsanity.com/paper/2102.11856/) (2021)
- [Domain Adaptation via Rebalanced Sub-domain Alignment](https://mlsanity.com/paper/2302.02009/) (2023)
- [Advances in Electron Microscopy with Deep Learning](https://mlsanity.com/paper/2101.01178/) (2021)
- [Employing chunk size adaptation to overcome concept drift](https://mlsanity.com/paper/2110.12881/) (2021)
- [Synergizing Deconfounding and Temporal Generalization For Time-series Counterfactual Outcome Estimation](https://mlsanity.com/paper/2511.16006/) (2025)

## Abstract (arXiv)

Generative Adversarial Networks (GANs) have proven to be a powerful framework for learning to draw samples from complex distributions. However, GANs are also notoriously difficult to train, with mode collapse and oscillations a common problem. We hypothesize that this is at least in part due to the evolution of the generator distribution and the catastrophic forgetting tendency of neural networks, which leads to the discriminator losing the ability to remember synthesized samples from previous instantiations of the generator. Recognizing this, our contributions are twofold. First, we show that GAN training makes for a more interesting and realistic benchmark for continual learning methods evaluation than some of the more canonical datasets. Second, we propose leveraging continual learning techniques to augment the discriminator, preserving its ability to recognize previous generator samples. We show that the resulting methods add only a light amount of computation, involve minimal changes to the model, and result in better overall performance on the examined image and text generation tasks.