# Deep, Skinny Neural Networks are not Universal Approximators

arXiv: https://arxiv.org/abs/1810.00393 · HTML: https://mlsanity.com/paper/1810.00393/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2018.

## Built on by (most cited first, 30 of 38)

- [Achieve the Minimum Width of Neural Networks for Universal Approximation](https://mlsanity.com/paper/2209.11395/) (2022)
- [The Real Tropical Geometry of Neural Networks](https://mlsanity.com/paper/2403.11871/) (2024)
- [A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models](https://mlsanity.com/paper/2401.07187/) (2024)
- [Minimum width for universal approximation using ReLU networks on compact domain](https://mlsanity.com/paper/2309.10402/) (2023)
- [A Survey on Universal Approximation Theorems](https://mlsanity.com/paper/2407.12895/) (2024)
- [Minimum Width of Leaky-ReLU Neural Networks for Uniform Universal Approximation](https://mlsanity.com/paper/2305.18460/) (2023)
- [Minimal Width for Universal Property of Deep RNN](https://mlsanity.com/paper/2211.13866/) (2022)
- [TNet: A Model-Constrained Tikhonov Network Approach for Inverse Problems](https://mlsanity.com/paper/2105.12033/) (2021)
- [Width is Less Important than Depth in ReLU Neural Networks](https://mlsanity.com/paper/2202.03841/) (2022)
- [Origami in N dimensions: How feed-forward networks manufacture linear separability](https://mlsanity.com/paper/2203.11355/) (2022)
- [Addressing common misinterpretations of KART and UAT in neural network literature](https://mlsanity.com/paper/2408.16389/) (2024)
- [Vocabulary for Universal Approximation: A Linguistic Perspective of Mapping Compositions](https://mlsanity.com/paper/2305.12205/) (2023)
- [Topological Deep Learning: Classification Neural Networks](https://mlsanity.com/paper/2102.08354/) (2021)
- [Vanilla Feedforward Neural Networks as a Discretization of Dynamical Systems](https://mlsanity.com/paper/2209.10909/) (2022)
- [Approximation and interpolation of deep neural networks](https://mlsanity.com/paper/2304.10552/) (2023)
- [A Model-Constrained Tangent Slope Learning Approach for Dynamical Systems](https://mlsanity.com/paper/2208.04995/) (2022)
- [Minimum width for universal approximation using squashable activation functions](https://mlsanity.com/paper/2504.07371/) (2025)
- [Qualitative neural network approximation over R and C: Elementary proofs for analytic and polynomial activation](https://mlsanity.com/paper/2203.13410/) (2022)
- [Optimal Neural Network Approximation for High-Dimensional Continuous Functions](https://mlsanity.com/paper/2409.02363/) (2024)
- [Approximation properties of neural ODEs](https://mlsanity.com/paper/2503.15696/) (2025)
- [Minimum Width of Deep Narrow Networks for Universal Approximation](https://mlsanity.com/paper/2511.06837/) (2025)
- [LU decomposition and Toeplitz decomposition of a neural network](https://mlsanity.com/paper/2211.13935/) (2022)
- [Dynamic neuron approach to deep neural networks: Decoupling neurons for renormalization group analysis](https://mlsanity.com/paper/2410.00396/) (2024)
- [MLPs and KANs for data-driven learning in physical problems: A performance comparison](https://mlsanity.com/paper/2504.11397/) (2025)
- [Latent Space Topology Evolution in Multilayer Perceptrons](https://mlsanity.com/paper/2506.01569/) (2025)
- [Low-dimensional topology of deep neural networks](https://mlsanity.com/paper/2606.31856/) (2026)
- [Two Instances of Interpretable Neural Network for Universal Approximations](https://mlsanity.com/paper/2112.15026/) (2021)
- [Quantitative Universal Approximation Bounds for Deep Belief Networks](https://mlsanity.com/paper/2208.09033/) (2022)
- [A Structural Approach to the Design of Domain Specific Neural Network Architectures](https://mlsanity.com/paper/2301.09381/) (2023)
- [Development of Non-Linear Equations for Predicting Electrical Conductivity in Silicates](https://mlsanity.com/paper/2305.13519/) (2023)

## Abstract (arXiv)

In order to choose a neural network architecture that will be effective for a particular modeling problem, one must understand the limitations imposed by each of the potential options. These limitations are typically described in terms of information theoretic bounds, or by comparing the relative complexity needed to approximate example functions between different architectures. In this paper, we examine the topological constraints that the architecture of a neural network imposes on the level sets of all the functions that it is able to approximate. This approach is novel for both the nature of the limitations and the fact that they are independent of network depth for a broad family of activation functions.