mlsanityData to 14 Oct 2026

Deep, Skinny Neural Networks are not Universal Approximators

arXiv 1810.00393 ↗First posted 30 Sep 2018cs.LG stat.MLcited by 38

Not on the map. Why?

Abstract (arXiv)

In order to choose a neural network architecture that will be effective for a particular modeling problem, one must understand the limitations imposed by each of the potential options. These limitations are typically described in terms of information theoretic bounds, or by comparing the relative complexity needed to approximate example functions between different architectures. In this paper, we examine the topological constraints that the architecture of a neural network imposes on the level sets of all the functions that it is able to approximate. This approach is novel for both the nature of the limitations and the fact that they are independent of network depth for a broad family of activation functions.

Key facts. Cited by 38 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 Sep 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 9; 2023: 6; 2024: 6; 2025: 7; 2026: 7.

38 papers.

Sort:

  1. 2022Achieve the Minimum Width of Neural Networks for Universal Approximationcited by 26
  2. 2024The Real Tropical Geometry of Neural Networkscited by 24
  3. 2024A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Modelscited by 20
  4. 2023Minimum width for universal approximation using ReLU networks on compact domaincited by 18
  5. 2024A Survey on Universal Approximation Theoremscited by 15
  6. 2023Minimum Width of Leaky-ReLU Neural Networks for Uniform Universal Approximationcited by 13
  7. 2022Minimal Width for Universal Property of Deep RNNcited by 12
  8. 2021TNet: A Model-Constrained Tikhonov Network Approach for Inverse Problemscited by 10
  9. 2022Width is Less Important than Depth in ReLU Neural Networkscited by 10
  10. 2022Origami in N dimensions: How feed-forward networks manufacture linear separabilitycited by 9
  11. 2024Addressing common misinterpretations of KART and UAT in neural network literaturecited by 9
  12. 2023Vocabulary for Universal Approximation: A Linguistic Perspective of Mapping Compositionscited by 6
26 more
  1. 2021Topological Deep Learning: Classification Neural Networkscited by 5
  2. 2022Vanilla Feedforward Neural Networks as a Discretization of Dynamical Systemscited by 4
  3. 2023Approximation and interpolation of deep neural networkscited by 4
  4. 2022A Model-Constrained Tangent Slope Learning Approach for Dynamical Systemscited by 3
  5. 2025Minimum width for universal approximation using squashable activation functionscited by 3
  6. 2022Qualitative neural network approximation over R and C: Elementary proofs for analytic and polynomial activationcited by 2
  7. 2024Optimal Neural Network Approximation for High-Dimensional Continuous Functionscited by 2
  8. 2025Approximation properties of neural ODEscited by 2
  9. 2025Minimum Width of Deep Narrow Networks for Universal Approximationcited by 2
  10. 2022LU decomposition and Toeplitz decomposition of a neural networkcited by 1
  11. 2024Dynamic neuron approach to deep neural networks: Decoupling neurons for renormalization group analysiscited by 1
  12. 2025MLPs and KANs for data-driven learning in physical problems: A performance comparisoncited by 1
  13. 2025Latent Space Topology Evolution in Multilayer Perceptronscited by 1
  14. 2026Low-dimensional topology of deep neural networkscited by 1
  15. 2021Two Instances of Interpretable Neural Network for Universal Approximationscited by 0
  16. 2022Quantitative Universal Approximation Bounds for Deep Belief Networkscited by 0
  17. 2023A Structural Approach to the Design of Domain Specific Neural Network Architecturescited by 0
  18. 2023Development of Non-Linear Equations for Predicting Electrical Conductivity in Silicatescited by 0
  19. 2025Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networkscited by 0
  20. 2025Explicit neural network classifiers for non-separable datacited by 0
  21. 2026Universal Approximation Theorem for Input-Connected Multilayer Perceptronscited by 0
  22. 2026Universality of shallow and deep neural networks on non-Euclidean spacescited by 0
  23. 2026Approximation Theory for Neural Networks: Old and Newcited by 0
  24. 2026Minimum Block Width for Universal Approximation by Residual Neural Networks with Inner Width Onecited by 0
  25. 2026Approximation of solutions of parameter-dependent problems by residual neural networkscited by 0
  26. 2026The minimal width of neural networks and immersions of real projective spacecited by 0