# Learning Distributions by Generative Adversarial Networks: Approximation and Generalization

arXiv: https://arxiv.org/abs/2205.12601 · HTML: https://mlsanity.com/paper/2205.12601/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 97 references that are in the map.
- First posted on arXiv 25 May 2022.

## Builds on (most cited first, 16 of 16)

- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Generative Moment Matching Networks](https://mlsanity.com/paper/1502.02761/) (2015)
- [Generative Adversarial Network in Medical Imaging: A Review](https://mlsanity.com/paper/1809.07294/) (2018)
- [Loss landscapes and optimization in over-parameterized non-linear systems and neural networks](https://mlsanity.com/paper/2003.00307/) (2020)
- [Optimal approximation of piecewise smooth functions using deep ReLU neural networks](https://mlsanity.com/paper/1709.05289/) (2017)
- [Universality of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1805.10769/) (2018)
- [Deep Network Approximation Characterized by Number of Neurons](https://mlsanity.com/paper/1906.05497/) (2019)
- [Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect](https://mlsanity.com/paper/1803.01541/) (2018)
- [Limitations of the Lipschitz constant as a defense against adversarial examples](https://mlsanity.com/paper/1807.09705/) (2018)
- [Convergence and Concentration of Empirical Measures under Wasserstein Distance in Unbounded Functional Spaces](https://mlsanity.com/paper/1804.10556/) (2018)
- [Distribution Approximation and Statistical Estimation Guarantees of Generative Adversarial Networks](https://mlsanity.com/paper/2002.03938/) (2020)
- [Approximation bounds for norm constrained neural networks with applications to regression and GANs](https://mlsanity.com/paper/2201.09418/) (2022)
- [On Convergence and Stability of GANs](https://mlsanity.com/paper/1705.07215/) (2017)
- [Approximation in shift-invariant spaces with deep ReLU neural networks](https://mlsanity.com/paper/2005.11949/) (2020)

## Built on by (most cited first, 1 of 1)

- [Optimal rates of approximation by shallow ReLUk neural networks and applications to nonparametric regression](https://mlsanity.com/paper/2304.01561/) (2023)

## Abstract (arXiv)

We study how well generative adversarial networks (GAN) learn probability distributions from finite samples by analyzing the convergence rates of these models. Our analysis is based on a new oracle inequality that decomposes the estimation error of GAN into the discriminator and generator approximation errors, generalization error and optimization error. To estimate the discriminator approximation error, we establish error bounds on approximating H"older functions by ReLU neural networks, with explicit upper bounds on the Lipschitz constant of the network or norm constraint on the weights. For generator approximation error, we show that neural network can approximately transform a low-dimensional source distribution to a high-dimensional target distribution and bound such approximation error by the width and depth of neural network. Combining the approximation results with generalization bounds of neural networks from statistical learning theory, we establish the convergence rates of GANs in various settings, when the error is measured by a collection of integral probability metrics defined through H"older classes, including the Wasserstein distance as a special case. In particular, for distributions concentrated around a low-dimensional set, we show that the convergence rates of GANs do not depend on the high ambient dimension, but on the lower intrinsic dimension.