# Overcoming Mode Collapse with Adaptive Multi Adversarial Training

arXiv: https://arxiv.org/abs/2112.14406 · HTML: https://mlsanity.com/paper/2112.14406/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 47 references that are in the map.
- First posted on arXiv 29 Dec 2021.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [PathNet: Evolution Channels Gradient Descent in Super Neural Networks](https://mlsanity.com/paper/1701.08734/) (2017)
- [PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models](https://mlsanity.com/paper/2003.03808/) (2020)
- [Generative Adversarial Network in Medical Imaging: A Review](https://mlsanity.com/paper/1809.07294/) (2018)
- [Unrolled Generative Adversarial Networks](https://mlsanity.com/paper/1611.02163/) (2016)
- [VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning](https://mlsanity.com/paper/1705.07761/) (2017)
- [Seeing What a GAN Cannot Generate](https://mlsanity.com/paper/1910.11626/) (2019)
- [Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping](https://mlsanity.com/paper/1709.07857/) (2017)
- [PacGAN: The power of two samples in generative adversarial networks](https://mlsanity.com/paper/1712.04086/) (2017)
- [RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Real](https://mlsanity.com/paper/2006.09001/) (2020)
- [Generative Multi-Adversarial Networks](https://mlsanity.com/paper/1611.01673/) (2016)
- [Dual Discriminator Generative Adversarial Nets](https://mlsanity.com/paper/1709.03831/) (2017)
- [AdaGAN: Boosting Generative Models](https://mlsanity.com/paper/1701.02386/) (2017)
- [Self-supervised GAN: Analysis and Improvement with Multi-class Minimax Game](https://mlsanity.com/paper/1911.06997/) (2019)
- [Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks](https://mlsanity.com/paper/1912.13503/) (2019)
- [Generative Adversarial Network Training is a Continual Learning Problem](https://mlsanity.com/paper/1811.11083/) (2018)

## Built on by (most cited first, 3 of 3)

- [Synthetic Health-related Longitudinal Data with Mixed-type Variables Generated using Diffusion Models](https://mlsanity.com/paper/2303.12281/) (2023)
- [Combating Mode Collapse in GANs via Manifold Entropy Estimation](https://mlsanity.com/paper/2208.12055/) (2022)
- [Scaling-based Data Augmentation for Generative Models and its Theoretical Extension](https://mlsanity.com/paper/2410.20780/) (2024)

## Abstract (arXiv)

Generative Adversarial Networks (GANs) are a class of generative models used for various applications, but they have been known to suffer from the mode collapse problem, in which some modes of the target distribution are ignored by the generator. Investigative study using a new data generation procedure indicates that the mode collapse of the generator is driven by the discriminator's inability to maintain classification accuracy on previously seen samples, a phenomenon called Catastrophic Forgetting in continual learning. Motivated by this observation, we introduce a novel training procedure that adaptively spawns additional discriminators to remember previous modes of generation. On several datasets, we show that our training scheme can be plugged-in to existing GAN frameworks to mitigate mode collapse and improve standard metrics for GAN evaluation.