# Cumulant GAN

arXiv: https://arxiv.org/abs/2006.06625 · HTML: https://mlsanity.com/paper/2006.06625/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2020.

## Built on by (most cited first, 9 of 9)

- [Function-space regularized R'enyi divergences](https://mlsanity.com/paper/2210.04974/) (2022)
- [α-GAN: Convergence and Estimation Guarantees](https://mlsanity.com/paper/2205.06393/) (2022)
- [Renyi Fair Information Bottleneck for Image Classification](https://mlsanity.com/paper/2203.04950/) (2022)
- [Deep into The Domain Shift: Transfer Learning through Dependence Regularization](https://mlsanity.com/paper/2305.19499/) (2023)
- [Classification Utility, Fairness, and Compactness via Tunable Information Bottleneck and R'enyi Measures](https://mlsanity.com/paper/2206.10043/) (2022)
- [Federated Learning with Only Positive Labels by Exploring Label Correlations](https://mlsanity.com/paper/2404.15598/) (2024)
- [Generalized Dual Discriminator GANs](https://mlsanity.com/paper/2507.17684/) (2025)
- [Concentration Inequalities for Stochastic Optimization of Unbounded Objective Functions with Application to Denoising Score Matching](https://mlsanity.com/paper/2502.08628/) (2025)
- [Hellinger loss function for Generative Adversarial Networks](https://mlsanity.com/paper/2512.12267/) (2025)

## Abstract (arXiv)

In this paper, we propose a novel loss function for training Generative Adversarial Networks (GANs) aiming towards deeper theoretical understanding as well as improved stability and performance for the underlying optimization problem. The new loss function is based on cumulant generating functions giving rise to Cumulant GAN. Relying on a recently-derived variational formula, we show that the corresponding optimization problem is equivalent to R'enyi divergence minimization, thus offering a (partially) unified perspective of GAN losses: the R'enyi family encompasses Kullback-Leibler divergence (KLD), reverse KLD, Hellinger distance and chi2-divergence. Wasserstein GAN is also a member of cumulant GAN. In terms of stability, we rigorously prove the linear convergence of cumulant GAN to the Nash equilibrium for a linear discriminator, Gaussian distributions and the standard gradient descent ascent algorithm. Finally, we experimentally demonstrate that image generation is more robust relative to Wasserstein GAN and it is substantially improved in terms of both inception score and Fr'echet inception distance when both weaker and stronger discriminators are considered.