mlsanityData to 7 Oct 2026

On Deep Generative Models for Approximation and Estimation of Distributions on Manifolds

arXiv 2302.13183 ↗First posted 25 Feb 2023stat.ML cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Generative networks have experienced great empirical successes in distribution learning. Many existing experiments have demonstrated that generative networks can generate high-dimensional complex data from a low-dimensional easy-to-sample distribution. However, this phenomenon can not be justified by existing theories. The widely held manifold hypothesis speculates that real-world data sets, such as natural images and signals, exhibit low-dimensional geometric structures. In this paper, we take such low-dimensional data structures into consideration by assuming that data distributions are supported on a low-dimensional manifold. We prove statistical guarantees of generative networks under the Wasserstein-1 loss. We show that the Wasserstein-1 loss converges to zero at a fast rate depending on the intrinsic dimension instead of the ambient data dimension. Our theory leverages the low-dimensional geometric structures in data sets and justifies the practical power of generative networks. We require no smoothness assumptions on the data distribution which is desirable in practice.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 25 of its 51 references that are in the map. First posted on arXiv 25 Feb 2023.

Builds on

25 of 51 references.

References · 25

  1. 2022Generative Adversarial Networkscited by 5,603
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2016Improved Techniques for Training GANscited by 2,938
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  5. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  6. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  7. 2016Generative Adversarial Text to Image Synthesiscited by 735
  8. 2016Error bounds for approximations with deep ReLU networkscited by 567
  9. 2017Learning Generative Models with Sinkhorn Divergencescited by 310
  10. 2015Learning with a Wasserstein Losscited by 194
  11. 2017Generalization and Equilibrium in Generative Adversarial Nets (GANs)cited by 165
  12. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 95
13 more
  1. 2018Evolving Mario Levels in the Latent Space of a Deep Convolutional Generative Adversarial Networkcited by 76
  2. 2019Nonparametric Regression on Low-Dimensional Manifolds using Deep ReLU Networks : Function Approximation and Statistical Recoverycited by 73
  3. 2017On the Discrimination-Generalization Tradeoff in GANscited by 48
  4. 2021An error analysis of generative adversarial networks for learning distributionscited by 35
  5. 2019Smooth function approximation by deep neural networks with general activation functionscited by 29
  6. 2018Approximability of Discriminators Implies Diversity in GANscited by 28
  7. 2020Statistical guarantees for generative models without dominationcited by 27
  8. 2017Generative Adversarial Networks recover features in astrophysical images of galaxies beyond the deconvolution limitcited by 26
  9. 2021On the capacity of deep generative networks for approximating distributionscited by 19
  10. 2017How Well Can Generative Adversarial Networks Learn Densities: A Nonparametric Viewcited by 14
  11. 2020Generalization Properties of Optimal Transport GANs with Latent Distribution Learningcited by 13
  12. 2021A likelihood approach to nonparametric estimation of a singular distribution using deep generative modelscited by 12
  13. 2018On Computation and Generalization of GANs with Spectrum Controlcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 4; 2026: 3.

12 papers.

  1. 2024Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Datacited by 30
  2. 2024Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Studycited by 13
  3. 2025CrowdLLM: Building LLM-Based Digital Populations Augmented with Generative Modelscited by 5
  4. 2025Deep Generative Models: Complexity, Dimensionality, and Approximationcited by 4
  5. 2025Image Interpolation with Score-based Riemannian Metrics of Diffusion Modelscited by 4
  6. 2024A Statistical Analysis of Wasserstein Autoencoders for Intrinsically Low-dimensional Datacited by 3
  7. 2025Coefficient-to-Basis Network: A Fine-Tunable Operator Learning Framework for Inverse Problems with Adaptive Discretizations and Theoretical Guaranteescited by 3
  8. 2024Generative adversarial learning with optimal input dimension and its adaptive generator architecturecited by 1
  9. 2026Learning Theory of Transformers: Local-to-Global Approximation via Softmax Partition of Unitycited by 1
  10. 2024A Statistical Analysis of Deep Federated Learning for Intrinsically Low-dimensional Datacited by 0
  11. 2026Identifiable Deep Latent Variable Models for MNAR Datacited by 0
  12. 2026Geometric Decoupling: Diagnosing the Structural Instability of Latentcited by 0