# A Survey on Universal Approximation Theorems

arXiv: https://arxiv.org/abs/2407.12895 · HTML: https://mlsanity.com/paper/2407.12895/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 49 references that are in the map.
- First posted on arXiv 17 Jul 2024.

## Builds on (most cited first, 14 of 14)

- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [On the Expressive Power of Deep Neural Networks](https://mlsanity.com/paper/1606.05336/) (2016)
- [Universality of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1805.10769/) (2018)
- [Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Review](https://mlsanity.com/paper/1611.00740/) (2016)
- [Optimal Approximation Rate of ReLU Networks in terms of Width and Depth](https://mlsanity.com/paper/2103.00502/) (2021)
- [On the number of response regions of deep feed forward networks with piece-wise linear activations](https://mlsanity.com/paper/1312.6098/) (2013)
- [Minimum Width for Universal Approximation](https://mlsanity.com/paper/2006.08859/) (2020)
- [A survey on modern trainable activation functions](https://mlsanity.com/paper/2005.00817/) (2020)
- [ResNet with one-neuron hidden layers is a Universal Approximator](https://mlsanity.com/paper/1806.10909/) (2018)
- [Universal Approximation Theorems for Differentiable Geometric Deep Learning](https://mlsanity.com/paper/2101.05390/) (2021)
- [Deep, Skinny Neural Networks are not Universal Approximators](https://mlsanity.com/paper/1810.00393/) (2018)
- [Sumformer: Universal Approximation for Efficient Transformers](https://mlsanity.com/paper/2307.02301/) (2023)
- [On the approximation by single hidden layer feedforward neural networks with fixed weights](https://mlsanity.com/paper/1708.06219/) (2017)
- [Minimal Width for Universal Property of Deep RNN](https://mlsanity.com/paper/2211.13866/) (2022)

## Built on by (most cited first, 15 of 15)

- [Solving Empirical Bayes via Transformers](https://mlsanity.com/paper/2502.09844/) (2025)
- [Universal Approximation of Mean-Field Models via Transformers](https://mlsanity.com/paper/2410.16295/) (2024)
- [Weight-Space Linear Recurrent Neural Networks](https://mlsanity.com/paper/2506.01153/) (2025)
- [A Conformal Prediction Framework for Uncertainty Quantification in Physics-Informed Neural Networks](https://mlsanity.com/paper/2509.13717/) (2025)
- [Hybrid Least Squares/Gradient Descent Methods for DeepONets](https://mlsanity.com/paper/2508.15394/) (2025)
- [Understanding the role of autoencoders for stiff dynamical systems using information theory](https://mlsanity.com/paper/2503.06325/) (2025)
- [Are neural scaling laws leading quantum chemistry astray?](https://mlsanity.com/paper/2509.26397/) (2025)
- [Benchmarking Simulacra AI's Quantum Accurate Synthetic Data Generation for Chemical Sciences](https://mlsanity.com/paper/2511.07433/) (2025)
- [LiLaN: A Linear Latent Network as the Solution Operator for Real-Time Solutions to Stiff Nonlinear Ordinary Differential Equations](https://mlsanity.com/paper/2501.08423/) (2025)
- [Imaging at the quantum limit with convolutional neural networks](https://mlsanity.com/paper/2506.13488/) (2025)
- [A Study of Hybrid and Evolutionary Metaheuristics for Single Hidden Layer Feedforward Neural Network Architecture](https://mlsanity.com/paper/2506.15737/) (2025)
- [Developing Training Procedures for Piecewise-linear Spline Activation Functions in Neural Networks](https://mlsanity.com/paper/2509.18161/) (2025)
- [Performance Evaluation of Transfer Learning Based Medical Image Classification Techniques for Disease Detection](https://mlsanity.com/paper/2512.04397/) (2025)
- [Modular connectivity in neural networks emerges from Poisson noise-motivated regularisation, and promotes robustness and compositional generalisation](https://mlsanity.com/paper/2512.13707/) (2025)
- [MIND: Lightweight and Effective Memory Injection Defense for LLM Agents via Intent-Aware Information Bottleneck](https://mlsanity.com/paper/2607.28103/) (2026)

## Abstract (arXiv)

This paper discusses various theorems on the approximation capabilities of neural networks (NNs), which are known as universal approximation theorems (UATs). The paper gives a systematic overview of UATs starting from the preliminary results on function approximation, such as Taylor's theorem, Fourier's theorem, Weierstrass approximation theorem, Kolmogorov - Arnold representation theorem, etc. Theoretical and numerical aspects of UATs are covered from both arbitrary width and depth.