mlsanityData to 14 Oct 2026

Approximating Probability Distributions by ReLU Networks

arXiv 2101.09973 ↗First posted 25 Jan 2021cs.LG cs.IT math.ITcited by 0

Not on the map. Why?

Abstract (arXiv)

How many neurons are needed to approximate a target probability distribution using a neural network with a given input distribution and approximation error? This paper examines this question for the case when the input distribution is uniform, and the target distribution belongs to the class of histogram distributions. We obtain a new upper bound on the number of required neurons, which is strictly better than previously existing upper bounds. The key ingredient in this improvement is an efficient construction of the neural nets representing piecewise linear functions. We also obtain a lower bound on the minimum number of neurons needed to approximate the histogram distributions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 14 references that are in the map. First posted on arXiv 25 Jan 2021.

Builds on

11 of 14 references.

References · 11

  1. 2013Auto-Encoding Variational Bayescited by 4,053
  2. 2016Error bounds for approximations with deep ReLU networkscited by 574
  3. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 464
  4. 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 257
  5. 2016Benefits of depth in neural networkscited by 250
  6. 2019Deep Neural Network Approximation Theorycited by 93
  7. 2019Nonlinear Approximation and (Deep) ReLU Networkscited by 45
  8. 2017On the ability of neural nets to express distributionscited by 26
  9. 2020Constructive Universal High-Dimensional Distribution Generation through Deep ReLU Networkscited by 13
  10. 2018Size-Noise Tradeoffs in Generative Networkscited by 12
  11. 2018Bounds on the Approximation Power of Feedforward Neural Networkscited by 3

Built on by

Not cited yet.