# A Convex Duality Framework for GANs

arXiv: https://arxiv.org/abs/1810.11740 · HTML: https://mlsanity.com/paper/1810.11740/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2018.

## Built on by (most cited first, 15 of 15)

- [Hidden Convexity of Wasserstein GANs: Interpretable Generative Models with Closed-Form Solutions](https://mlsanity.com/paper/2107.05680/) (2021)
- [Parallel Deep Neural Networks Have Zero Duality Gap](https://mlsanity.com/paper/2110.06482/) (2021)
- [Toward a `Standard Model' of Machine Learning](https://mlsanity.com/paper/2108.07783/) (2021)
- [A Wasserstein Minimax Framework for Mixed Linear Regression](https://mlsanity.com/paper/2106.07537/) (2021)
- [Inferential Wasserstein Generative Adversarial Networks](https://mlsanity.com/paper/2109.05652/) (2021)
- [Moreau-Yosida f-divergences](https://mlsanity.com/paper/2102.13416/) (2021)
- [Generative Adversarial Networks and Adversarial Autoencoders: Tutorial and Survey](https://mlsanity.com/paper/2111.13282/) (2021)
- [Generative Minimization Networks: Training GANs Without Competition](https://mlsanity.com/paper/2103.12685/) (2021)
- [On Characterizing GAN Convergence Through Proximal Duality Gap](https://mlsanity.com/paper/2105.04801/) (2021)
- [A Wasserstein perspective of Vanilla GANs](https://mlsanity.com/paper/2403.15312/) (2024)
- [A Class-aware Optimal Transport Approach with Higher-Order Moment Matching for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2401.15952/) (2024)
- [Statistical Error Bounds for GANs with Nonlinear Objective Functionals](https://mlsanity.com/paper/2406.16834/) (2024)
- [Scalable Personalised Item Ranking through Parametric Density Estimation](https://mlsanity.com/paper/2105.04769/) (2021)
- [Dual feature-based and example-based explanation methods](https://mlsanity.com/paper/2401.16294/) (2024)
- [Seasoning Generative Models for a Generalization Aftertaste](https://mlsanity.com/paper/2603.18817/) (2026)

## Abstract (arXiv)

Generative adversarial network (GAN) is a minimax game between a generator mimicking the true model and a discriminator distinguishing the samples produced by the generator from the real training samples. Given an unconstrained discriminator able to approximate any function, this game reduces to finding the generative model minimizing a divergence measure, e.g. the Jensen-Shannon (JS) divergence, to the data distribution. However, in practice the discriminator is constrained to be in a smaller class F such as neural nets. Then, a natural question is how the divergence minimization interpretation changes as we constrain F. In this work, we address this question by developing a convex duality framework for analyzing GANs. For a convex set F, this duality framework interprets the original GAN formulation as finding the generative model with minimum JS-divergence to the distributions penalized to match the moments of the data distribution, with the moments specified by the discriminators in F. We show that this interpretation more generally holds for f-GAN and Wasserstein GAN. As a byproduct, we apply the duality framework to a hybrid of f-divergence and Wasserstein distance. Unlike the f-divergence, we prove that the proposed hybrid divergence changes continuously with the generative model, which suggests regularizing the discriminator's Lipschitz constant in f-GAN and vanilla GAN. We numerically evaluate the power of the suggested regularization schemes for improving GAN's training performance.