# Minimum Width of Deep Narrow Networks for Universal Approximation

arXiv: https://arxiv.org/abs/2511.06837 · HTML: https://mlsanity.com/paper/2511.06837/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 29 references that are in the map.
- First posted on arXiv 10 Nov 2025.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Universal Approximation with Deep Narrow Networks](https://mlsanity.com/paper/1905.08539/) (2019)
- [Optimal approximation of continuous functions by very deep ReLU networks](https://mlsanity.com/paper/1802.03620/) (2018)
- [Approximating Continuous Functions by ReLU Nets of Minimal Width](https://mlsanity.com/paper/1710.11278/) (2017)
- [Topology of deep neural networks](https://mlsanity.com/paper/2004.06093/) (2020)
- [Minimum Width for Universal Approximation](https://mlsanity.com/paper/2006.08859/) (2020)
- [On the Optimal Memorization Power of ReLU Neural Networks](https://mlsanity.com/paper/2110.03187/) (2021)
- [Deep, Skinny Neural Networks are not Universal Approximators](https://mlsanity.com/paper/1810.00393/) (2018)
- [Provable Memorization via Deep Neural Networks using Sub-linear Parameters](https://mlsanity.com/paper/2010.13363/) (2020)
- [Achieve the Minimum Width of Neural Networks for Universal Approximation](https://mlsanity.com/paper/2209.11395/) (2022)
- [Mathematical theory of deep learning](https://mlsanity.com/paper/2407.18384/) (2024)
- [The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective](https://mlsanity.com/paper/2106.06529/) (2021)
- [Minimum width for universal approximation using ReLU networks on compact domain](https://mlsanity.com/paper/2309.10402/) (2023)
- [Improved weight initialization for deep and narrow feedforward neural network](https://mlsanity.com/paper/2311.03733/) (2023)
- [Exact representation and efficient approximations of linear model predictive control laws via HardTanh type deep neural networks](https://mlsanity.com/paper/2401.05076/) (2024)
- [When narrower is better: the narrow width limit of Bayesian parallel branching neural networks](https://mlsanity.com/paper/2407.18807/) (2024)

## Built on by (most cited first, 2 of 2)

- [Relocation of compact sets in ℝn by diffeomorphisms and linear separability of datasets in ℝn](https://mlsanity.com/paper/2604.21393/) (2026)
- [Minimum Block Width for Universal Approximation by Residual Neural Networks with Inner Width One](https://mlsanity.com/paper/2607.04597/) (2026)

## Abstract (arXiv)

Determining the minimum width of fully connected neural networks has become a fundamental problem in recent theoretical studies of deep neural networks. In this paper, we study the lower bounds and upper bounds of the minimum width required for fully connected neural networks in order to have universal approximation capability, which is important in network design and training. We show that wmin≤max(2dx+1, dy) also holds true for networks with ELU, SELU activation functions, and the upper bound of this inequality is attained when dy=2dx, where dx, dy denote the input and output dimensions, respectively. Besides, we show that dx+1≤ wmin≤ dx+dy for networks with LeakyReLU, ELU, CELU, SELU, Softplus activation functions, by proving that ReLU activation function can be approximated by these activation functions. In addition, in the case that the activation function is injective or can be uniformly approximated by a sequence of injective functions (e.g., ReLU), we present a new proof of the inequality wmin≥ dy+1dx<dy≤2dx by constructing a more intuitive example via a new geometric approach based on Poincar'e-Miranda Theorem.