# Minimum width for universal approximation using ReLU networks on compact domain arXiv: https://arxiv.org/abs/2309.10402 · HTML: https://mlsanity.com/paper/2309.10402/ ## Key facts - Cited by 18 papers in the map (data through 14 Oct 2026). - Builds on 20 of its 36 references that are in the map. - First posted on arXiv 19 Sep 2023. ## Builds on (most cited first, 20 of 20) - [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021) - [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017) - [Benefits of depth in neural networks](https://mlsanity.com/paper/1602.04485/) (2016) - [Universal Approximation with Deep Narrow Networks](https://mlsanity.com/paper/1905.08539/) (2019) - [The Power of Depth for Feedforward Neural Networks](https://mlsanity.com/paper/1512.03965/) (2015) - [Optimal approximation of continuous functions by very deep ReLU networks](https://mlsanity.com/paper/1802.03620/) (2018) - [Approximating Continuous Functions by ReLU Nets of Minimal Width](https://mlsanity.com/paper/1710.11278/) (2017) - [Minimum Width for Universal Approximation](https://mlsanity.com/paper/2006.08859/) (2020) - [Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity](https://mlsanity.com/paper/1810.07770/) (2018) - [Deep Learning via Dynamical Systems: An Approximation Perspective](https://mlsanity.com/paper/1912.10382/) (2019) - [Depth Separation for Neural Networks](https://mlsanity.com/paper/1702.08489/) (2017) - [On the Optimal Memorization Power of ReLU Neural Networks](https://mlsanity.com/paper/2110.03187/) (2021) - [Deep, Skinny Neural Networks are not Universal Approximators](https://mlsanity.com/paper/1810.00393/) (2018) - [Provable Memorization via Deep Neural Networks using Sub-linear Parameters](https://mlsanity.com/paper/2010.13363/) (2020) - [Achieve the Minimum Width of Neural Networks for Universal Approximation](https://mlsanity.com/paper/2209.11395/) (2022) - [Minimal Width for Universal Property of Deep RNN](https://mlsanity.com/paper/2211.13866/) (2022) - [Approximation capabilities of neural networks on unbounded domains](https://mlsanity.com/paper/1910.09293/) (2019) - [Vanilla Feedforward Neural Networks as a Discretization of Dynamical Systems](https://mlsanity.com/paper/2209.10909/) (2022) - [A Note on the Representation Power of GHHs](https://mlsanity.com/paper/2101.11286/) (2021) - [Minimum Width for Deep, Narrow MLP: A Diffeomorphism Approach](https://mlsanity.com/paper/2308.15873/) (2023) ## Built on by (most cited first, 18 of 18) - [Understanding the Difficulty of Solving Cauchy Problems with PINNs](https://mlsanity.com/paper/2405.02561/) (2024) - [On Expressive Power of Quantized Neural Networks under Fixed-Point Arithmetic](https://mlsanity.com/paper/2409.00297/) (2024) - [Minimum width for universal approximation using squashable activation functions](https://mlsanity.com/paper/2504.07371/) (2025) - [Deep Ridgelet Transform and Unified Universality Theorem for Deep and Shallow Joint-Group-Equivariant Machines](https://mlsanity.com/paper/2405.13682/) (2024) - [Optimal Neural Network Approximation for High-Dimensional Continuous Functions](https://mlsanity.com/paper/2409.02363/) (2024) - [Constructive Universal Approximation and Finite Sample Memorization by Narrow Deep ReLU Networks](https://mlsanity.com/paper/2409.06555/) (2024) - [Minimum Width of Deep Narrow Networks for Universal Approximation](https://mlsanity.com/paper/2511.06837/) (2025) - [Blessing of Dimensionality for Approximating Sobolev Classes on Manifolds](https://mlsanity.com/paper/2408.06996/) (2024) - [Extended Fiducial Inference for Individual Treatment Effects via Deep Neural Networks](https://mlsanity.com/paper/2505.01995/) (2025) - [Approximation and learning with compositional tensor trains](https://mlsanity.com/paper/2512.18059/) (2025) - [Low-dimensional topology of deep neural networks](https://mlsanity.com/paper/2606.31856/) (2026) - [Sparse FEONet: A Low-Cost, Memory-Efficient Operator Network via Finite-Element Local Sparsity for Parametric PDEs](https://mlsanity.com/paper/2601.00672/) (2026) - [Geometric separation and constructive universal approximation with two hidden layers](https://mlsanity.com/paper/2602.12482/) (2026) - [The minimal width of universal p-adic ReLU neural networks](https://mlsanity.com/paper/2603.00064/) (2026) - [Classification Fields: Arbitrarily Fine Recursive Hierarchical Clustering From Few Examples](https://mlsanity.com/paper/2605.07119/) (2026) - [Approximation Theory for Neural Networks: Old and New](https://mlsanity.com/paper/2605.21451/) (2026) - [Minimum Block Width for Universal Approximation by Residual Neural Networks with Inner Width One](https://mlsanity.com/paper/2607.04597/) (2026) - [The minimal width of neural networks and immersions of real projective space](https://mlsanity.com/paper/2610.20229/) (2026) ## Abstract (arXiv) It has been shown that deep neural networks of a large enough width are universal approximators but they are not if the width is too small. There were several attempts to characterize the minimum width wmin enabling the universal approximation property; however, only a few of them found the exact values. In this work, we show that the minimum width for Lp approximation of Lp functions from [0,1]dx to mathbb Rdy is exactly maxdx,dy,2 if an activation function is ReLU-Like (e.g., ReLU, GELU, Softplus). Compared to the known result for ReLU networks, wmin=maxdx+1,dy when the domain is smashmathbb Rdx, our result first shows that approximation on a compact domain requires smaller width than on smashmathbb Rdx. We next prove a lower bound on wmin for uniform approximation using general activation functions including ReLU: wmin≥ dy+1 if dx