# Size-Noise Tradeoffs in Generative Networks

arXiv: https://arxiv.org/abs/1810.11158 · HTML: https://mlsanity.com/paper/1810.11158/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Oct 2018.

## Built on by (most cited first, 12 of 12)

- [Normalizing flow neural networks by JKO scheme](https://mlsanity.com/paper/2212.14424/) (2022)
- [An error analysis of generative adversarial networks for learning distributions](https://mlsanity.com/paper/2105.13010/) (2021)
- [On the capacity of deep generative networks for approximating distributions](https://mlsanity.com/paper/2101.12353/) (2021)
- [Universal Approximation of Residual Flows in Maximum Mean Discrepancy](https://mlsanity.com/paper/2103.05793/) (2021)
- [High-Dimensional Distribution Generation Through Deep Neural Networks](https://mlsanity.com/paper/2107.12466/) (2021)
- [Generative neural networks for characteristic functions](https://mlsanity.com/paper/2401.04778/) (2024)
- [Finding mixed-strategy equilibria of continuous-action games without gradients using randomized policy networks](https://mlsanity.com/paper/2211.15936/) (2022)
- [Generative adversarial learning with optimal input dimension and its adaptive generator architecture](https://mlsanity.com/paper/2405.03723/) (2024)
- [Approximating Probability Distributions by ReLU Networks](https://mlsanity.com/paper/2101.09973/) (2021)
- [Quantitative Universal Approximation Bounds for Deep Belief Networks](https://mlsanity.com/paper/2208.09033/) (2022)
- [Learnability of high-dimensional targets by two-parameter models and gradient flow](https://mlsanity.com/paper/2402.17089/) (2024)
- [On optimal solutions of classical and sliced Wasserstein GANs with non-Gaussian data](https://mlsanity.com/paper/2509.06505/) (2025)

## Abstract (arXiv)

This paper investigates the ability of generative networks to convert their input noise distributions into other distributions. Firstly, we demonstrate a construction that allows ReLU networks to increase the dimensionality of their noise distribution by implementing a "space-filling" function based on iterated tent maps. We show this construction is optimal by analyzing the number of affine pieces in functions computed by multivariate ReLU networks. Secondly, we provide efficient ways (using polylog (1/ε) nodes) for networks to pass between univariate uniform and normal distributions, using a Taylor series approximation and a binary search gadget for computing function inverses. Lastly, we indicate how high dimensional distributions can be efficiently transformed into low dimensional distributions.