mlsanityData to 7 Oct 2026

A Survey on Universal Approximation Theorems

arXiv 2407.12895 ↗First posted 17 Jul 2024cs.LG cs.SY eess.SYcited by 15

Not on the map. Why?

Abstract (arXiv)

This paper discusses various theorems on the approximation capabilities of neural networks (NNs), which are known as universal approximation theorems (UATs). The paper gives a systematic overview of UATs starting from the preliminary results on function approximation, such as Taylor's theorem, Fourier's theorem, Weierstrass approximation theorem, Kolmogorov - Arnold representation theorem, etc. Theoretical and numerical aspects of UATs are covered from both arbitrary width and depth.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 14 of its 49 references that are in the map. First posted on arXiv 17 Jul 2024.

Builds on

14 of 49 references.

References · 14

  1. 2017The Expressive Power of Neural Networks: A View from the Widthcited by 318
  2. 2016On the Expressive Power of Deep Neural Networkscited by 302
  3. 2018Universality of Deep Convolutional Neural Networkscited by 176
  4. 2016Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Reviewcited by 155
  5. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 95
  6. 2013On the number of response regions of deep feed forward networks with piece-wise linear activationscited by 94
  7. 2020Minimum Width for Universal Approximationcited by 92
  8. 2020A survey on modern trainable activation functionscited by 89
  9. 2018ResNet with one-neuron hidden layers is a Universal Approximatorcited by 78
  10. 2021Universal Approximation Theorems for Differentiable Geometric Deep Learningcited by 41
  11. 2018Deep, Skinny Neural Networks are not Universal Approximatorscited by 37
  12. 2023Sumformer: Universal Approximation for Efficient Transformerscited by 30
2 more
  1. 2017On the approximation by single hidden layer feedforward neural networks with fixed weightscited by 19
  2. 2022Minimal Width for Universal Property of Deep RNNcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 13; 2026: 1.

15 papers.

Sort:

By area:

  1. 2024Universal Approximation of Mean-Field Models via Transformerscited by 7
  2. 2025Solving Empirical Bayes via Transformerscited by 7
  3. 2025Weight-Space Linear Recurrent Neural Networkscited by 5
  4. 2025A Conformal Prediction Framework for Uncertainty Quantification in Physics-Informed Neural Networkscited by 5
  5. 2025Hybrid Least Squares/Gradient Descent Methods for DeepONetscited by 2
  6. 2025Understanding the role of autoencoders for stiff dynamical systems using information theorycited by 1
  7. 2025Are neural scaling laws leading quantum chemistry astray?cited by 1
  8. 2025LiLaN: A Linear Latent Network as the Solution Operator for Real-Time Solutions to Stiff Nonlinear Ordinary Differential Equationscited by 0
  9. 2025Imaging at the quantum limit with convolutional neural networkscited by 0
  10. 2025A Study of Hybrid and Evolutionary Metaheuristics for Single Hidden Layer Feedforward Neural Network Architecturecited by 0
  11. 2025Developing Training Procedures for Piecewise-linear Spline Activation Functions in Neural Networkscited by 0
  12. 2025Benchmarking Simulacra AI's Quantum Accurate Synthetic Data Generation for Chemical Sciencescited by 0
3 more
  1. 2025Performance Evaluation of Transfer Learning Based Medical Image Classification Techniques for Disease Detectioncited by 0
  2. 2025Modular connectivity in neural networks emerges from Poisson noise-motivated regularisation, and promotes robustness and compositional generalisationcited by 0
  3. 2026MIND: Lightweight and Effective Memory Injection Defense for LLM Agents via Intent-Aware Information Bottleneckcited by 0