mlsanityData to 14 Oct 2026

On the capacity of deep generative networks for approximating distributions

arXiv 2101.12353 ↗First posted 29 Jan 2021cs.LG math.PR math.STcited by 19

Not on the map. Why?

Abstract (arXiv)

We study the efficacy and efficiency of deep generative networks for approximating probability distributions. We prove that neural networks can transform a low-dimensional source distribution to a distribution that is arbitrarily close to a high-dimensional target distribution, when the closeness are measured by Wasserstein distances and maximum mean discrepancy. Upper bounds of the approximation error are obtained in terms of the width and depth of neural network. Furthermore, it is shown that the approximation error in Wasserstein distance grows at most linearly on the ambient dimension and that the approximation order only depends on the intrinsic dimension of the target distribution. On the contrary, when f-divergences are used as metrics of distributions, the approximation property is different. We show that in order to approximate the target distribution in f-divergences, the dimension of the source distribution cannot be smaller than the intrinsic dimension of the target distribution.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 21 of its 57 references that are in the map. First posted on arXiv 29 Jan 2021.

Builds on

21 of 57 references.

References · 21

  1. 2013Auto-Encoding Variational Bayescited by 4,053
  2. 2018Demystifying MMD GANscited by 1,293
  3. 2016Generative Adversarial Text to Image Synthesiscited by 735
  4. 2017Wasserstein GANcited by 639
  5. 2015Generating Sentences from a Continuous Spacecited by 612
  6. 2016Error bounds for approximations with deep ReLU networkscited by 574
  7. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 506
  8. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 464
  9. 2015Generative Moment Matching Networkscited by 257
  10. 2018Generative Adversarial Network in Medical Imaging: A Reviewcited by 257
  11. 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 179
  12. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 166
9 more
  1. 2020Deep Network Approximation for Smooth Functionscited by 164
  2. 2015Training generative neural networks via Maximum Mean Discrepancy optimizationcited by 151
  3. 2019Deep Network Approximation Characterized by Number of Neuronscited by 122
  4. 2019The phase diagram of approximation rates for deep neural networkscited by 88
  5. 2018How Well Generative Adversarial Networks Learn Distributionscited by 78
  6. 2018Convergence and Concentration of Empirical Measures under Wasserstein Distance in Unbounded Functional Spacescited by 41
  7. 2017On the ability of neural nets to express distributionscited by 26
  8. 2020Constructive Universal High-Dimensional Distribution Generation through Deep ReLU Networkscited by 13
  9. 2018Size-Noise Tradeoffs in Generative Networkscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 3; 2023: 7; 2024: 2; 2025: 1; 2026: 1.

19 papers.

Sort:

By area:

  1. 2021An error analysis of generative adversarial networks for learning distributionscited by 36
  2. 2023Convergence of flow-based generative models via proximal gradient descent in Wasserstein spacecited by 34
  3. 2021Wasserstein Generative Learning of Conditional Distributioncited by 24
  4. 2022Approximation bounds for norm constrained neural networks with applications to regression and GANscited by 22
  5. 2023Complexity Matters: Rethinking the Latent Space for Generative Modelingcited by 19
  6. 2021Wasserstein Generative Adversarial Uncertainty Quantification in Physics-Informed Neural Networkscited by 16
  7. 2023On Deep Generative Models for Approximation and Estimation of Distributions on Manifoldscited by 13
  8. 2023Neural Operator Variational Inference based on Regularized Stein Discrepancy for Deep Gaussian Processescited by 6
  9. 2023Statistical Guarantees of Group-Invariant GANscited by 4
  10. 2025Deep Generative Models: Complexity, Dimensionality, and Approximationcited by 4
  11. 2021Non-Asymptotic Error Bounds for Bidirectional GANscited by 3
  12. 2023Statistically Optimal Generative Modeling with Maximum Deviation from the Empirical Distributioncited by 3
7 more
  1. 2024Generative neural networks for characteristic functionscited by 3
  2. 2023Probabilistic Invariant Learning with Randomized Linear Classifierscited by 2
  3. 2024Generative adversarial learning with optimal input dimension and its adaptive generator architecturecited by 1
  4. 2021Approximating Probability Distributions by using Wasserstein Generative Adversarial Networkscited by 0
  5. 2022Distribution estimation and change-point estimation for time series via DNN-based GANscited by 0
  6. 2022Distribution Estimation of Contaminated Data via DNN-based MoM-GANscited by 0
  7. 2026Learning stochastic multiscale models through normalizing flowscited by 0