# New advances in universal approximation with neural networks of minimal width

arXiv: https://arxiv.org/abs/2411.08735 · HTML: https://mlsanity.com/paper/2411.08735/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Nov 2024.

## Built on by (most cited first, 5 of 5)

- [On Information Geometry and Iterative Optimization in Model Compression: Operator Factorization](https://mlsanity.com/paper/2507.09428/) (2025)
- [Low-dimensional topology of deep neural networks](https://mlsanity.com/paper/2606.31856/) (2026)
- [Consistency of Learned Sparse Grid Quadrature Rules using NeuralODEs](https://mlsanity.com/paper/2507.01533/) (2025)
- [Approximation Theory for Neural Networks: Old and New](https://mlsanity.com/paper/2605.21451/) (2026)
- [The minimal width of neural networks and immersions of real projective space](https://mlsanity.com/paper/2610.20229/) (2026)

## Abstract (arXiv)

We prove several universal approximation results at minimal or near-minimal width for approximation of Lp(ℝdx, ℝdy) and C0(ℝdx, ℝdy) on compact sets. Our approach uses a unified coding scheme that yields explicit constructions relying only on standard analytic tools. We show that feedforward neural networks with two leaky ReLU activations σα, σ-α achieve the optimal width maxdx, dy for Lp approximation, while a single leaky ReLU σα achieves width max2, dx, dy, providing an alternative proof of the results of Cai et al. (2023). By generalizing to stepped leaky ReLU activations, we extend these results to uniform approximation of continuous functions while identifying sets of activation functions compatible with gradient-based training. Since our constructions pass through an intermediate dimension of one, they imply that autoencoders with a one-dimensional feature space are universal approximators. We further show that squashable activations combined with FLOOR achieve width max3, dx, dy for uniform approximation. We also establish a lower bound of maxdx, dy + 1 for networks when all activations are continuous and monotone and dy ≤ 2dx. Moreover, we extend our results to invertible LU-decomposable networks, proving distributional universal approximation for LU-Net normalizing flows and providing a constructive proof of the classical theorem of Brenier and Gangbo on Lp approximation by diffeomorphisms.