# On Deep Generative Models for Approximation and Estimation of Distributions on Manifolds

arXiv: https://arxiv.org/abs/2302.13183 · HTML: https://mlsanity.com/paper/2302.13183/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 51 references that are in the map.
- First posted on arXiv 25 Feb 2023.

## Builds on (most cited first, 25 of 25)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Learning Generative Models with Sinkhorn Divergences](https://mlsanity.com/paper/1706.00292/) (2017)
- [Learning with a Wasserstein Loss](https://mlsanity.com/paper/1506.05439/) (2015)
- [Generalization and Equilibrium in Generative Adversarial Nets (GANs)](https://mlsanity.com/paper/1703.00573/) (2017)
- [Optimal Approximation Rate of ReLU Networks in terms of Width and Depth](https://mlsanity.com/paper/2103.00502/) (2021)
- [Evolving Mario Levels in the Latent Space of a Deep Convolutional Generative Adversarial Network](https://mlsanity.com/paper/1805.00728/) (2018)
- [Nonparametric Regression on Low-Dimensional Manifolds using Deep ReLU Networks : Function Approximation and Statistical Recovery](https://mlsanity.com/paper/1908.01842/) (2019)
- [On the Discrimination-Generalization Tradeoff in GANs](https://mlsanity.com/paper/1711.02771/) (2017)
- [An error analysis of generative adversarial networks for learning distributions](https://mlsanity.com/paper/2105.13010/) (2021)
- [Smooth function approximation by deep neural networks with general activation functions](https://mlsanity.com/paper/1906.06903/) (2019)
- [Approximability of Discriminators Implies Diversity in GANs](https://mlsanity.com/paper/1806.10586/) (2018)
- [Statistical guarantees for generative models without domination](https://mlsanity.com/paper/2010.09237/) (2020)
- [Generative Adversarial Networks recover features in astrophysical images of galaxies beyond the deconvolution limit](https://mlsanity.com/paper/1702.00403/) (2017)
- [On the capacity of deep generative networks for approximating distributions](https://mlsanity.com/paper/2101.12353/) (2021)
- [How Well Can Generative Adversarial Networks Learn Densities: A Nonparametric View](https://mlsanity.com/paper/1712.08244/) (2017)
- [Generalization Properties of Optimal Transport GANs with Latent Distribution Learning](https://mlsanity.com/paper/2007.14641/) (2020)
- [A likelihood approach to nonparametric estimation of a singular distribution using deep generative models](https://mlsanity.com/paper/2105.04046/) (2021)
- [On Computation and Generalization of GANs with Spectrum Control](https://mlsanity.com/paper/1812.10912/) (2018)

## Built on by (most cited first, 13 of 13)

- [Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data](https://mlsanity.com/paper/2411.06646/) (2024)
- [Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Study](https://mlsanity.com/paper/2410.00357/) (2024)
- [CrowdLLM: Building LLM-Based Digital Populations Augmented with Generative Models](https://mlsanity.com/paper/2512.07890/) (2025)
- [A Statistical Analysis of Wasserstein Autoencoders for Intrinsically Low-dimensional Data](https://mlsanity.com/paper/2402.15710/) (2024)
- [Deep Generative Models: Complexity, Dimensionality, and Approximation](https://mlsanity.com/paper/2504.00820/) (2025)
- [Image Interpolation with Score-based Riemannian Metrics of Diffusion Models](https://mlsanity.com/paper/2504.20288/) (2025)
- [Coefficient-to-Basis Network: A Fine-Tunable Operator Learning Framework for Inverse Problems with Adaptive Discretizations and Theoretical Guarantees](https://mlsanity.com/paper/2503.08642/) (2025)
- [Learning Theory of Transformers: Local-to-Global Approximation via Softmax Partition of Unity](https://mlsanity.com/paper/2605.08811/) (2026)
- [Generative adversarial learning with optimal input dimension and its adaptive generator architecture](https://mlsanity.com/paper/2405.03723/) (2024)
- [Generalization Bounds for Flow-matching Generative Models for Intrinsically Low-dimensional Data](https://mlsanity.com/paper/2610.02663/) (2026)
- [A Statistical Analysis of Deep Federated Learning for Intrinsically Low-dimensional Data](https://mlsanity.com/paper/2410.20659/) (2024)
- [Identifiable Deep Latent Variable Models for MNAR Data](https://mlsanity.com/paper/2603.24771/) (2026)
- [Geometric Decoupling: Diagnosing the Structural Instability of Latent](https://mlsanity.com/paper/2604.18804/) (2026)

## Abstract (arXiv)

Generative networks have experienced great empirical successes in distribution learning. Many existing experiments have demonstrated that generative networks can generate high-dimensional complex data from a low-dimensional easy-to-sample distribution. However, this phenomenon can not be justified by existing theories. The widely held manifold hypothesis speculates that real-world data sets, such as natural images and signals, exhibit low-dimensional geometric structures. In this paper, we take such low-dimensional data structures into consideration by assuming that data distributions are supported on a low-dimensional manifold. We prove statistical guarantees of generative networks under the Wasserstein-1 loss. We show that the Wasserstein-1 loss converges to zero at a fast rate depending on the intrinsic dimension instead of the ambient data dimension. Our theory leverages the low-dimensional geometric structures in data sets and justifies the practical power of generative networks. We require no smoothness assumptions on the data distribution which is desirable in practice.