New advances in universal approximation with neural networks of minimal width

We prove several universal approximation results at minimal or near-minimal width for approximation of Lp(ℝdx, ℝdy) and C0(ℝdx, ℝdy) on compact sets. Our approach uses a unified coding scheme that yields explicit constructions relying only on standard analytic tools. We show that feedforward neural networks with two leaky ReLU activations σα, σ-α achieve the optimal width maxdx, dy for Lp approximation, while a single leaky ReLU σα achieves width max2, dx, dy, providing an alternative proof of the results of Cai et al. (2023). By generalizing to stepped leaky ReLU activations, we extend these results to uniform approximation of continuous functions while identifying sets of activation functions compatible with gradient-based training. Since our constructions pass through an intermediate dimension of one, they imply that autoencoders with a one-dimensional feature space are universal approximators. We further show that squashable activations combined with FLOOR achieve width max3, dx, dy for uniform approximation. We also establish a lower bound of maxdx, dy + 1 for networks when all activations are continuous and monotone and dy ≤ 2dx. Moreover, we extend our results to invertible LU-decomposable networks, proving distributional universal approximation for LU-Net normalizing flows and providing a constructive proof of the classical theorem of Brenier and Gangbo on Lp approximation by diffeomorphisms.