# Addressing GAN Training Instabilities via Tunable Classification Losses

arXiv: https://arxiv.org/abs/2310.18291 · HTML: https://mlsanity.com/paper/2310.18291/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 69 references that are in the map.
- First posted on arXiv 27 Oct 2023.

## Builds on (most cited first, 30 of 43)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks](https://mlsanity.com/paper/1602.07868/) (2016)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [Adversarial Feature Learning](https://mlsanity.com/paper/1605.09782/) (2016)
- [Norm-Based Capacity Control in Neural Networks](https://mlsanity.com/paper/1503.00036/) (2015)
- [Size-Independent Sample Complexity of Neural Networks](https://mlsanity.com/paper/1712.06541/) (2017)
- [Generative Moment Matching Networks](https://mlsanity.com/paper/1502.02761/) (2015)
- [MMD GAN: Towards Deeper Understanding of Moment Matching Network](https://mlsanity.com/paper/1705.08584/) (2017)
- [VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning](https://mlsanity.com/paper/1705.07761/) (2017)
- [Generalization and Equilibrium in Generative Adversarial Nets (GANs)](https://mlsanity.com/paper/1703.00573/) (2017)
- [Training generative neural networks via Maximum Mean Discrepancy optimization](https://mlsanity.com/paper/1505.03906/) (2015)
- [BEGAN: Boundary Equilibrium Generative Adversarial Networks](https://mlsanity.com/paper/1703.10717/) (2017)
- [PacGAN: The power of two samples in generative adversarial networks](https://mlsanity.com/paper/1712.04086/) (2017)
- [Bias and Generalization in Deep Generative Models: An Empirical Study](https://mlsanity.com/paper/1811.03259/) (2018)
- [How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?](https://mlsanity.com/paper/1511.05101/) (2015)
- [How Well Generative Adversarial Networks Learn Distributions](https://mlsanity.com/paper/1811.03179/) (2018)
- [Stabilizing Generative Adversarial Networks: A Survey](https://mlsanity.com/paper/1910.00927/) (2019)
- [On the Discrimination-Generalization Tradeoff in GANs](https://mlsanity.com/paper/1711.02771/) (2017)
- [Approximation and Convergence Properties of Generative Adversarial Learning](https://mlsanity.com/paper/1705.08991/) (2017)
- [A Systematic Survey of Regularization and Normalization in GANs](https://mlsanity.com/paper/2008.08930/) (2020)
- [Outlier-Robust Optimal Transport: Duality, Structure, and Statistical Analysis](https://mlsanity.com/paper/2111.01361/) (2021)

## Built on by (most cited first, 5 of 5)

- [Query2CAD: Generating CAD models using natural language queries](https://mlsanity.com/paper/2406.00144/) (2024)
- [Generating Realistic X-ray Scattering Images Using Stable Diffusion and Human-in-the-loop Annotations](https://mlsanity.com/paper/2408.12720/) (2024)
- [Generalized Dual Discriminator GANs](https://mlsanity.com/paper/2507.17684/) (2025)
- [α-GAN by R'enyi Cross Entropy](https://mlsanity.com/paper/2505.14190/) (2025)
- [Bounds on the Excess Minimum Risk via Generalized Information Divergence Measures](https://mlsanity.com/paper/2505.24117/) (2025)

## Abstract (arXiv)

Generative adversarial networks (GANs), modeled as a zero-sum game between a generator (G) and a discriminator (D), allow generating synthetic data with formal guarantees. Noting that D is a classifier, we begin by reformulating the GAN value function using class probability estimation (CPE) losses. We prove a two-way correspondence between CPE loss GANs and f-GANs which minimize f-divergences. We also show that all symmetric f-divergences are equivalent in convergence. In the finite sample and model capacity setting, we define and obtain bounds on estimation and generalization errors. We specialize these results to α-GANs, defined using α-loss, a tunable CPE loss family parametrized by α∈(0,∞]. We next introduce a class of dual-objective GANs to address training instabilities of GANs by modeling each player's objective using α-loss to obtain (αD,αG)-GANs. We show that the resulting non-zero sum game simplifies to minimizing an f-divergence under appropriate conditions on (αD,αG). Generalizing this dual-objective formulation using CPE losses, we define and obtain upper bounds on an appropriately defined estimation error. Finally, we highlight the value of tuning (αD,αG) in alleviating training instabilities for the synthetic 2D Gaussian mixture ring as well as the large publicly available Celeb-A and LSUN Classroom image datasets.