# How Well Generative Adversarial Networks Learn Distributions

arXiv: https://arxiv.org/abs/1811.03179 · HTML: https://mlsanity.com/paper/1811.03179/

## Key facts

- Cited by 78 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2018.

## Built on by (most cited first, 30 of 78)

- [An error analysis of generative adversarial networks for learning distributions](https://mlsanity.com/paper/2105.13010/) (2021)
- [Toward Understanding Generative Data Augmentation](https://mlsanity.com/paper/2305.17476/) (2023)
- [Approximation bounds for norm constrained neural networks with applications to regression and GANs](https://mlsanity.com/paper/2201.09418/) (2022)
- [A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models](https://mlsanity.com/paper/2401.07187/) (2024)
- [On the capacity of deep generative networks for approximating distributions](https://mlsanity.com/paper/2101.12353/) (2021)
- [Complexity Matters: Rethinking the Latent Space for Generative Modeling](https://mlsanity.com/paper/2307.08283/) (2023)
- [Gaussian Interpolation Flows](https://mlsanity.com/paper/2311.11475/) (2023)
- [Wasserstein Generative Adversarial Uncertainty Quantification in Physics-Informed Neural Networks](https://mlsanity.com/paper/2108.13054/) (2021)
- [Constrained Policy Optimization with Explicit Behavior Density for Offline Reinforcement Learning](https://mlsanity.com/paper/2301.12130/) (2023)
- [Statistical Guarantees for Variational Autoencoders using PAC-Bayesian Theory](https://mlsanity.com/paper/2310.04935/) (2023)
- [A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loops](https://mlsanity.com/paper/2502.18865/) (2025)
- [Rates of convergence for density estimation with generative adversarial networks](https://mlsanity.com/paper/2102.00199/) (2021)
- [PAC-Bayesian Generalization Bounds for Adversarial Generative Models](https://mlsanity.com/paper/2302.08942/) (2023)
- [A likelihood approach to nonparametric estimation of a singular distribution using deep generative models](https://mlsanity.com/paper/2105.04046/) (2021)
- [Sample Complexity Bounds for Estimating Probability Divergences under Invariances](https://mlsanity.com/paper/2311.02868/) (2023)
- [Realizing GANs via a Tunable Loss Function](https://mlsanity.com/paper/2106.05232/) (2021)
- [Reversible Gromov-Monge Sampler for Simulation-Based Inference](https://mlsanity.com/paper/2109.14090/) (2021)
- [On the minimax optimality of Flow Matching through the connection to kernel density estimation](https://mlsanity.com/paper/2504.13336/) (2025)
- [Testing for the Markov Property in Time Series via Deep Conditional Generative Learning](https://mlsanity.com/paper/2305.19244/) (2023)
- [Utility Theory of Synthetic Data Generation](https://mlsanity.com/paper/2305.10015/) (2023)
- [Causality Pursuit from Heterogeneous Environments via Neural Adversarial Invariance Learning](https://mlsanity.com/paper/2405.04715/) (2024)
- [Minimax Optimality (Probably) Doesn't Imply Distribution Learning for GANs](https://mlsanity.com/paper/2201.07206/) (2022)
- [Online Learning to Transport via the Minimal Selection Principle](https://mlsanity.com/paper/2202.04732/) (2022)
- [Generative Bayesian Inference with GANs](https://mlsanity.com/paper/2208.12113/) (2022)
- [Statistical guarantees for denoising reflected diffusion models](https://mlsanity.com/paper/2411.01563/) (2024)
- [Distribution estimation via Flow Matching with Lipschitz guarantees](https://mlsanity.com/paper/2509.02337/) (2025)
- [A Manifold Two-Sample Test Study: Integral Probability Metric with Neural Networks](https://mlsanity.com/paper/2205.02043/) (2022)
- [α-GAN: Convergence and Estimation Guarantees](https://mlsanity.com/paper/2205.06393/) (2022)
- [Addressing GAN Training Instabilities via Tunable Classification Losses](https://mlsanity.com/paper/2310.18291/) (2023)
- [Denoising Diffusions with Optimal Transport: Localization, Curvature, and Multi-Scale Complexity](https://mlsanity.com/paper/2411.01629/) (2024)

## Abstract (arXiv)

This paper studies the rates of convergence for learning distributions implicitly with the adversarial framework and Generative Adversarial Networks (GANs), which subsume Wasserstein, Sobolev, MMD GAN, and Generalized/Simulated Method of Moments (GMM/SMM) as special cases. We study a wide range of parametric and nonparametric target distributions under a host of objective evaluation metrics. We investigate how to obtain valid statistical guarantees for GANs through the lens of regularization. On the nonparametric end, we derive the optimal minimax rates for distribution estimation under the adversarial framework. On the parametric end, we establish a theory for general neural network classes (including deep leaky ReLU networks) that characterizes the interplay on the choice of generator and discriminator pair. We discover and isolate a new notion of regularization, called the generator-discriminator-pair regularization, that sheds light on the advantage of GANs compared to classical parametric and nonparametric approaches for explicit distribution estimation. We develop novel oracle inequalities as the main technical tools for analyzing GANs, which are of independent interest.