# Deconstructing Generative Adversarial Networks

arXiv: https://arxiv.org/abs/1901.09465 · HTML: https://mlsanity.com/paper/1901.09465/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jan 2019.

## Built on by (most cited first, 15 of 15)

- [Byzantine-Robust Federated Learning with Optimal Statistical Rates and Privacy Guarantees](https://mlsanity.com/paper/2205.11765/) (2022)
- [Robust W-GAN-Based Estimation Under Wasserstein Contamination](https://mlsanity.com/paper/2101.07969/) (2021)
- [On the 1-Wasserstein Distance between Location-Scale Distributions and the Effect of Differential Privacy](https://mlsanity.com/paper/2304.14869/) (2023)
- [GANs as Gradient Flows that Converge](https://mlsanity.com/paper/2205.02910/) (2022)
- [Pursuit of a Discriminative Representation for Multiple Subspaces via Sequential Games](https://mlsanity.com/paper/2206.09120/) (2022)
- [Robust Estimation for Nonparametric Families via Generative Adversarial Networks](https://mlsanity.com/paper/2202.01269/) (2022)
- [Generative Adversarial Network: Some Analytical Perspectives](https://mlsanity.com/paper/2104.12210/) (2021)
- [Blind Estimation of a Doubly Selective OFDM Channel: A Deep Learning Algorithm and Theory](https://mlsanity.com/paper/2206.07483/) (2022)
- [Optimal Robust Estimation under Local and Global Corruptions: Stronger Adversary and Smaller Error](https://mlsanity.com/paper/2410.17230/) (2024)
- [Attacking Byzantine Robust Aggregation in High Dimensions](https://mlsanity.com/paper/2312.14461/) (2023)
- [Robust Estimation for Kernel Exponential Families with Smoothed Total Variation Distances](https://mlsanity.com/paper/2410.20760/) (2024)
- [Distribution estimation and change-point estimation for time series via DNN-based GANs](https://mlsanity.com/paper/2211.14577/) (2022)
- [Distribution Estimation of Contaminated Data via DNN-based MoM-GANs](https://mlsanity.com/paper/2212.13741/) (2022)
- [Insights into Closed-form IPM-GAN Discriminator Guidance for Diffusion Modeling](https://mlsanity.com/paper/2306.01654/) (2023)
- [Fundamental Limits of Deep Learning-Based Binary Classifiers Trained with Hinge Loss](https://mlsanity.com/paper/2309.06774/) (2023)

## Abstract (arXiv)

We deconstruct the performance of GANs into three components: 1. Formulation: we propose a perturbation view of the population target of GANs. Building on this interpretation, we show that GANs can be viewed as a generalization of the robust statistics framework, and propose a novel GAN architecture, termed as Cascade GANs, to provably recover meaningful low-dimensional generator approximations when the real distribution is high-dimensional and corrupted by outliers. 2. Generalization: given a population target of GANs, we design a systematic principle, projection under admissible distance, to design GANs to meet the population requirement using finite samples. We implement our principle in three cases to achieve polynomial and sometimes near-optimal sample complexities: (1) learning an arbitrary generator under an arbitrary pseudonorm; (2) learning a Gaussian location family under TV distance, where we utilize our principle provide a new proof for the optimality of Tukey median viewed as GANs; (3) learning a low-dimensional Gaussian approximation of a high-dimensional arbitrary distribution under Wasserstein distance. We demonstrate a fundamental trade-off in the approximation error and statistical error in GANs, and show how to apply our principle with empirical samples to predict how many samples are sufficient for GANs in order not to suffer from the discriminator winning problem. 3. Optimization: we demonstrate alternating gradient descent is provably not locally asymptotically stable in optimizing the GAN formulation of PCA. We diagnose the problem as the minimax duality gap being non-zero, and propose a new GAN architecture whose duality gap is zero, where the value of the game is equal to the previous minimax value (not the maximin value). We prove the new GAN architecture is globally asymptotically stable in optimization under alternating gradient descent.