# Constructive Universal High-Dimensional Distribution Generation through Deep ReLU Networks

arXiv: https://arxiv.org/abs/2006.16664 · HTML: https://mlsanity.com/paper/2006.16664/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2020.

## Built on by (most cited first, 13 of 13)

- [Normalizing flow neural networks by JKO scheme](https://mlsanity.com/paper/2212.14424/) (2022)
- [An error analysis of generative adversarial networks for learning distributions](https://mlsanity.com/paper/2105.13010/) (2021)
- [Approximation bounds for norm constrained neural networks with applications to regression and GANs](https://mlsanity.com/paper/2201.09418/) (2022)
- [On the capacity of deep generative networks for approximating distributions](https://mlsanity.com/paper/2101.12353/) (2021)
- [Latent State Marginalization as a Low-cost Approach for Improving Exploration](https://mlsanity.com/paper/2210.00999/) (2022)
- [Universal Approximation of Residual Flows in Maximum Mean Discrepancy](https://mlsanity.com/paper/2103.05793/) (2021)
- [Neural Operator Variational Inference based on Regularized Stein Discrepancy for Deep Gaussian Processes](https://mlsanity.com/paper/2309.12658/) (2023)
- [High-Dimensional Distribution Generation Through Deep Neural Networks](https://mlsanity.com/paper/2107.12466/) (2021)
- [Data Topology-Dependent Upper Bounds of Neural Network Widths](https://mlsanity.com/paper/2305.16375/) (2023)
- [Generative adversarial learning with optimal input dimension and its adaptive generator architecture](https://mlsanity.com/paper/2405.03723/) (2024)
- [Approximating Probability Distributions by ReLU Networks](https://mlsanity.com/paper/2101.09973/) (2021)
- [Quantitative Universal Approximation Bounds for Deep Belief Networks](https://mlsanity.com/paper/2208.09033/) (2022)
- [Learnability of high-dimensional targets by two-parameter models and gradient flow](https://mlsanity.com/paper/2402.17089/) (2024)

## Abstract (arXiv)

We present an explicit deep neural network construction that transforms uniformly distributed one-dimensional noise into an arbitrarily close approximation of any two-dimensional Lipschitz-continuous target distribution. The key ingredient of our design is a generalization of the "space-filling" property of sawtooth functions discovered in (Bailey & Telgarsky, 2018). We elicit the importance of depth - in our neural network construction - in driving the Wasserstein distance between the target distribution and the approximation realized by the network to zero. An extension to output distributions of arbitrary dimension is outlined. Finally, we show that the proposed construction does not incur a cost - in terms of error measured in Wasserstein-distance - relative to generating d-dimensional target distributions from d independent random variables.