mlsanityData to 14 Oct 2026

Minimum width for universal approximation using ReLU networks on compact domain

arXiv 2309.10402 ↗First posted 19 Sep 2023cs.LG stat.MLcited by 18

Not on the map. Why?

Abstract (arXiv)

It has been shown that deep neural networks of a large enough width are universal approximators but they are not if the width is too small. There were several attempts to characterize the minimum width wmin enabling the universal approximation property; however, only a few of them found the exact values. In this work, we show that the minimum width for Lp approximation of Lp functions from [0,1]dx to mathbb Rdy is exactly maxdx,dy,2 if an activation function is ReLU-Like (e.g., ReLU, GELU, Softplus). Compared to the known result for ReLU networks, wmin=maxdx+1,dy when the domain is smashmathbb Rdx, our result first shows that approximation on a compact domain requires smaller width than on smashmathbb Rdx. We next prove a lower bound on wmin for uniform approximation using general activation functions including ReLU: wmin≥ dy+1 if dx<dy≤2dx. Together with our first result, this shows a dichotomy between Lp and uniform approximations for general activation functions and input/output dimensions.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 20 of its 36 references that are in the map. First posted on arXiv 19 Sep 2023.

Builds on

20 of 36 references.

References · 20

  1. 2021Convex optimizationcited by 2,515
  2. 2017The Expressive Power of Neural Networks: A View from the Widthcited by 320
  3. 2016Benefits of depth in neural networkscited by 250
  4. 2019Universal Approximation with Deep Narrow Networkscited by 200
  5. 2015The Power of Depth for Feedforward Neural Networkscited by 183
  6. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 166
  7. 2017Approximating Continuous Functions by ReLU Nets of Minimal Widthcited by 122
  8. 2020Minimum Width for Universal Approximationcited by 94
  9. 2018Small ReLU networks are powerful memorizers: a tight analysis of memorization capacitycited by 78
  10. 2019Deep Learning via Dynamical Systems: An Approximation Perspectivecited by 78
  11. 2017Depth Separation for Neural Networkscited by 46
  12. 2021On the Optimal Memorization Power of ReLU Neural Networkscited by 40
8 more
  1. 2018Deep, Skinny Neural Networks are not Universal Approximatorscited by 38
  2. 2020Provable Memorization via Deep Neural Networks using Sub-linear Parameterscited by 34
  3. 2022Achieve the Minimum Width of Neural Networks for Universal Approximationcited by 26
  4. 2022Minimal Width for Universal Property of Deep RNNcited by 12
  5. 2019Approximation capabilities of neural networks on unbounded domainscited by 7
  6. 2022Vanilla Feedforward Neural Networks as a Discretization of Dynamical Systemscited by 4
  7. 2021A Note on the Representation Power of GHHscited by 3
  8. 2023Minimum Width for Deep, Narrow MLP: A Diffeomorphism Approachcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 4; 2026: 8.

18 papers.

Sort:

  1. 2024Understanding the Difficulty of Solving Cauchy Problems with PINNscited by 6
  2. 2024On Expressive Power of Quantized Neural Networks under Fixed-Point Arithmeticcited by 4
  3. 2025Minimum width for universal approximation using squashable activation functionscited by 3
  4. 2024Deep Ridgelet Transform and Unified Universality Theorem for Deep and Shallow Joint-Group-Equivariant Machinescited by 2
  5. 2024Optimal Neural Network Approximation for High-Dimensional Continuous Functionscited by 2
  6. 2024Constructive Universal Approximation and Finite Sample Memorization by Narrow Deep ReLU Networkscited by 2
  7. 2025Minimum Width of Deep Narrow Networks for Universal Approximationcited by 2
  8. 2024Blessing of Dimensionality for Approximating Sobolev Classes on Manifoldscited by 1
  9. 2025Extended Fiducial Inference for Individual Treatment Effects via Deep Neural Networkscited by 1
  10. 2025Approximation and learning with compositional tensor trainscited by 1
  11. 2026Low-dimensional topology of deep neural networkscited by 1
  12. 2026Sparse FEONet: A Low-Cost, Memory-Efficient Operator Network via Finite-Element Local Sparsity for Parametric PDEscited by 0
6 more
  1. 2026Geometric separation and constructive universal approximation with two hidden layerscited by 0
  2. 2026The minimal width of universal p-adic ReLU neural networkscited by 0
  3. 2026Classification Fields: Arbitrarily Fine Recursive Hierarchical Clustering From Few Examplescited by 0
  4. 2026Approximation Theory for Neural Networks: Old and Newcited by 0
  5. 2026Minimum Block Width for Universal Approximation by Residual Neural Networks with Inner Width Onecited by 0
  6. 2026The minimal width of neural networks and immersions of real projective spacecited by 0