# Quantitative Universal Approximation Bounds for Deep Belief Networks

arXiv: https://arxiv.org/abs/2208.09033 · HTML: https://mlsanity.com/paper/2208.09033/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 66 references that are in the map.
- First posted on arXiv 18 Aug 2022.

## Builds on (most cited first, 16 of 16)

- [Distributional Reinforcement Learning with Quantile Regression](https://mlsanity.com/paper/1710.10044/) (2017)
- [On the number of response regions of deep feed forward networks with piece-wise linear activations](https://mlsanity.com/paper/1312.6098/) (2013)
- [Deep, Skinny Neural Networks are not Universal Approximators](https://mlsanity.com/paper/1810.00393/) (2018)
- [Flexible Model Aggregation for Quantile Regression](https://mlsanity.com/paper/2103.00083/) (2021)
- [Restricted Boltzmann Machines: Introduction and Review](https://mlsanity.com/paper/1806.07066/) (2018)
- [Constructive Universal High-Dimensional Distribution Generation through Deep ReLU Networks](https://mlsanity.com/paper/2006.16664/) (2020)
- [Size-Noise Tradeoffs in Generative Networks](https://mlsanity.com/paper/1810.11158/) (2018)
- [Stochastic Feedforward Neural Networks: Universal Approximation](https://mlsanity.com/paper/1910.09763/) (2019)
- [An Adaptive Learning Method of Deep Belief Network by Layer Generation Algorithm](https://mlsanity.com/paper/1807.03486/) (2018)
- [Restricted Boltzmann Machine and Deep Belief Network: Tutorial and Survey](https://mlsanity.com/paper/2107.12521/) (2021)
- [Discrete Restricted Boltzmann Machines](https://mlsanity.com/paper/1301.3529/) (2013)
- [DeepQA: Improving the estimation of single protein model quality with deep belief networks](https://mlsanity.com/paper/1607.04379/) (2016)
- [Deep Adversarial Belief Networks](https://mlsanity.com/paper/1909.06134/) (2019)
- [Deep Narrow Boltzmann Machines are Universal Approximators](https://mlsanity.com/paper/1411.3784/) (2014)
- [Universal Approximation of Markov Kernels by Shallow Stochastic Feedforward Networks](https://mlsanity.com/paper/1503.07211/) (2015)
- [An Object Detection by using Adaptive Structural Learning of Deep Belief Network](https://mlsanity.com/paper/1909.13465/) (2019)

## Abstract (arXiv)

We show that deep belief networks with binary hidden units can approximate any multivariate probability density under very mild integrability requirements on the parental density of the visible nodes. The approximation is measured in the Lq-norm for q∈[1,∞] (q=∞ corresponding to the supremum norm) and in Kullback-Leibler divergence. Furthermore, we establish sharp quantitative bounds on the approximation error in terms of the number of hidden units.