# Universal Approximation Theorems of Fully Connected Binarized Neural Networks

arXiv: https://arxiv.org/abs/2102.02631 · HTML: https://mlsanity.com/paper/2102.02631/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 4 Feb 2021.

## Builds on (most cited first, 10 of 10)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations](https://mlsanity.com/paper/1609.07061/) (2016)
- [Scalable and accurate deep learning for electronic health records](https://mlsanity.com/paper/1801.07860/) (2018)
- [Training Quantized Nets: A Deeper Understanding](https://mlsanity.com/paper/1706.02379/) (2017)
- [Towards the Limit of Network Quantization](https://mlsanity.com/paper/1612.01543/) (2016)
- [On the Universal Approximability and Complexity Bounds of Quantized ReLU Neural Networks](https://mlsanity.com/paper/1802.03646/) (2018)
- [How Does Batch Normalization Help Binary Training?](https://mlsanity.com/paper/1909.09139/) (2019)
- [Additive Noise Annealing and Approximation Properties of Quantized Neural Networks](https://mlsanity.com/paper/1905.10452/) (2019)
- [On the Universal Approximation Property and Equivalence of Stochastic Computing-based Neural Networks and Binary Neural Networks](https://mlsanity.com/paper/1803.05391/) (2018)

## Built on by (most cited first, 2 of 2)

- [Closing the Theory-Practice Gap in Spiking Transformers via Effective Dimension](https://mlsanity.com/paper/2604.15769/) (2026)
- [On the Expressive Power of Weight Quantization in Large Language Models](https://mlsanity.com/paper/2606.22249/) (2026)

## Abstract (arXiv)

Neural networks (NNs) are known for their high predictive accuracy in complex learning problems. Beside practical advantages, NNs also indicate favourable theoretical properties such as universal approximation (UA) theorems. Binarized Neural Networks (BNNs) significantly reduce time and memory demands by restricting the weight and activation domains to two values. Despite the practical advantages, theoretical guarantees based on UA theorems of BNNs are rather sparse in the literature. We close this gap by providing UA theorems for fully connected BNNs under the following scenarios: (1) for binarized inputs, UA can be constructively achieved under one hidden layer; (2) for inputs with real numbers, UA can not be achieved under one hidden layer but can be constructively achieved under two hidden layers for Lipschitz-continuous functions. Our results indicate that fully connected BNNs can approximate functions universally, under certain conditions.