mlsanityData to 14 Oct 2026

Minimum Width of Deep Narrow Networks for Universal Approximation

arXiv 2511.06837 ↗First posted 10 Nov 2025cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Determining the minimum width of fully connected neural networks has become a fundamental problem in recent theoretical studies of deep neural networks. In this paper, we study the lower bounds and upper bounds of the minimum width required for fully connected neural networks in order to have universal approximation capability, which is important in network design and training. We show that wmin≤max(2dx+1, dy) also holds true for networks with ELU, SELU activation functions, and the upper bound of this inequality is attained when dy=2dx, where dx, dy denote the input and output dimensions, respectively. Besides, we show that dx+1≤ wmin≤ dx+dy for networks with LeakyReLU, ELU, CELU, SELU, Softplus activation functions, by proving that ReLU activation function can be approximated by these activation functions. In addition, in the case that the activation function is injective or can be uniformly approximated by a sequence of injective functions (e.g., ReLU), we present a new proof of the inequality wmin≥ dy+1dx<dy≤2dx by constructing a more intuitive example via a new geometric approach based on Poincar'e-Miranda Theorem.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 17 of its 29 references that are in the map. First posted on arXiv 10 Nov 2025.

Builds on

17 of 29 references.

References · 17

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  3. 2019Universal Approximation with Deep Narrow Networkscited by 200
  4. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 166
  5. 2017Approximating Continuous Functions by ReLU Nets of Minimal Widthcited by 122
  6. 2020Topology of deep neural networkscited by 112
  7. 2020Minimum Width for Universal Approximationcited by 94
  8. 2021On the Optimal Memorization Power of ReLU Neural Networkscited by 40
  9. 2018Deep, Skinny Neural Networks are not Universal Approximatorscited by 38
  10. 2020Provable Memorization via Deep Neural Networks using Sub-linear Parameterscited by 34
  11. 2022Achieve the Minimum Width of Neural Networks for Universal Approximationcited by 26
  12. 2024Mathematical theory of deep learningcited by 23
5 more
  1. 2021The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspectivecited by 21
  2. 2023Minimum width for universal approximation using ReLU networks on compact domaincited by 18
  3. 2023Improved weight initialization for deep and narrow feedforward neural networkcited by 7
  4. 2024Exact representation and efficient approximations of linear model predictive control laws via HardTanh type deep neural networkscited by 4
  5. 2024When narrower is better: the narrow width limit of Bayesian parallel branching neural networkscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Relocation of compact sets in ℝn by diffeomorphisms and linear separability of datasets in ℝncited by 0
  2. 2026Minimum Block Width for Universal Approximation by Residual Neural Networks with Inner Width Onecited by 0