# On the capacity of deep generative networks for approximating distributions

arXiv: https://arxiv.org/abs/2101.12353 · HTML: https://mlsanity.com/paper/2101.12353/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 57 references that are in the map.
- First posted on arXiv 29 Jan 2021.

## Builds on (most cited first, 21 of 21)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [Generative Moment Matching Networks](https://mlsanity.com/paper/1502.02761/) (2015)
- [Generative Adversarial Network in Medical Imaging: A Review](https://mlsanity.com/paper/1809.07294/) (2018)
- [Optimal approximation of piecewise smooth functions using deep ReLU neural networks](https://mlsanity.com/paper/1709.05289/) (2017)
- [Optimal approximation of continuous functions by very deep ReLU networks](https://mlsanity.com/paper/1802.03620/) (2018)
- [Deep Network Approximation for Smooth Functions](https://mlsanity.com/paper/2001.03040/) (2020)
- [Training generative neural networks via Maximum Mean Discrepancy optimization](https://mlsanity.com/paper/1505.03906/) (2015)
- [Deep Network Approximation Characterized by Number of Neurons](https://mlsanity.com/paper/1906.05497/) (2019)
- [The phase diagram of approximation rates for deep neural networks](https://mlsanity.com/paper/1906.09477/) (2019)
- [How Well Generative Adversarial Networks Learn Distributions](https://mlsanity.com/paper/1811.03179/) (2018)
- [Convergence and Concentration of Empirical Measures under Wasserstein Distance in Unbounded Functional Spaces](https://mlsanity.com/paper/1804.10556/) (2018)
- [On the ability of neural nets to express distributions](https://mlsanity.com/paper/1702.07028/) (2017)
- [Constructive Universal High-Dimensional Distribution Generation through Deep ReLU Networks](https://mlsanity.com/paper/2006.16664/) (2020)
- [Size-Noise Tradeoffs in Generative Networks](https://mlsanity.com/paper/1810.11158/) (2018)

## Built on by (most cited first, 19 of 19)

- [An error analysis of generative adversarial networks for learning distributions](https://mlsanity.com/paper/2105.13010/) (2021)
- [Convergence of flow-based generative models via proximal gradient descent in Wasserstein space](https://mlsanity.com/paper/2310.17582/) (2023)
- [Wasserstein Generative Learning of Conditional Distribution](https://mlsanity.com/paper/2112.10039/) (2021)
- [Approximation bounds for norm constrained neural networks with applications to regression and GANs](https://mlsanity.com/paper/2201.09418/) (2022)
- [Complexity Matters: Rethinking the Latent Space for Generative Modeling](https://mlsanity.com/paper/2307.08283/) (2023)
- [Wasserstein Generative Adversarial Uncertainty Quantification in Physics-Informed Neural Networks](https://mlsanity.com/paper/2108.13054/) (2021)
- [On Deep Generative Models for Approximation and Estimation of Distributions on Manifolds](https://mlsanity.com/paper/2302.13183/) (2023)
- [Neural Operator Variational Inference based on Regularized Stein Discrepancy for Deep Gaussian Processes](https://mlsanity.com/paper/2309.12658/) (2023)
- [Statistical Guarantees of Group-Invariant GANs](https://mlsanity.com/paper/2305.13517/) (2023)
- [Deep Generative Models: Complexity, Dimensionality, and Approximation](https://mlsanity.com/paper/2504.00820/) (2025)
- [Non-Asymptotic Error Bounds for Bidirectional GANs](https://mlsanity.com/paper/2110.12319/) (2021)
- [Statistically Optimal Generative Modeling with Maximum Deviation from the Empirical Distribution](https://mlsanity.com/paper/2307.16422/) (2023)
- [Generative neural networks for characteristic functions](https://mlsanity.com/paper/2401.04778/) (2024)
- [Probabilistic Invariant Learning with Randomized Linear Classifiers](https://mlsanity.com/paper/2308.04412/) (2023)
- [Generative adversarial learning with optimal input dimension and its adaptive generator architecture](https://mlsanity.com/paper/2405.03723/) (2024)
- [Approximating Probability Distributions by using Wasserstein Generative Adversarial Networks](https://mlsanity.com/paper/2103.10060/) (2021)
- [Distribution estimation and change-point estimation for time series via DNN-based GANs](https://mlsanity.com/paper/2211.14577/) (2022)
- [Distribution Estimation of Contaminated Data via DNN-based MoM-GANs](https://mlsanity.com/paper/2212.13741/) (2022)
- [Learning stochastic multiscale models through normalizing flows](https://mlsanity.com/paper/2605.09718/) (2026)

## Abstract (arXiv)

We study the efficacy and efficiency of deep generative networks for approximating probability distributions. We prove that neural networks can transform a low-dimensional source distribution to a distribution that is arbitrarily close to a high-dimensional target distribution, when the closeness are measured by Wasserstein distances and maximum mean discrepancy. Upper bounds of the approximation error are obtained in terms of the width and depth of neural network. Furthermore, it is shown that the approximation error in Wasserstein distance grows at most linearly on the ambient dimension and that the approximation order only depends on the intrinsic dimension of the target distribution. On the contrary, when f-divergences are used as metrics of distributions, the approximation property is different. We show that in order to approximate the target distribution in f-divergences, the dimension of the source distribution cannot be smaller than the intrinsic dimension of the target distribution.