Arbitrary-Accuracy Neural Approximation with Optimal Neuron Count and Near-Optimal Bit Complexity
Not on the map. Why?
Abstract (arXiv)
We study the minimum number of hidden neurons required for arbitrary-accuracy approximation of multivariate H"older-continuous functions on [0,1]d and the associated encoding complexity. For d≥ 2, we construct a fixed, explicitly defined activation function for which a closed-form network with two hidden layers of widths d and 1 achieves arbitrary accuracy in the uniform norm. We prove that d+1 is the exact minimum total number of hidden neurons among standard feedforward networks with locally integrable activations and affine outputs. We further give a simpler construction using a single elementary activation that combines the floor and exponential functions. This construction requires three hidden layers of widths d, 1, and 2, only two neurons above the minimum. If a skip connection is allowed, widths d, 1, and 1 suffice. These constructions use explicit grid addressing and integer encoding of quantized function values. For a bounded α-H"older class, they require O(varepsilon-d/αlog(1/varepsilon)) bits, matching the metric-entropy lower bound up to a logarithmic factor.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 19 references that are in the map. First posted on arXiv 28 Sep 2026.
Builds on
9 of 19 references.
References · 9
- 2016Error bounds for approximations with deep ReLU networkscited by 574
- 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 179
- 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 166
- 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 96
- 2020Neural Network Approximation: Three Hidden Layers Are Enoughcited by 63
- 2021Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neuronscited by 41
- 2021Elementary superexpressive activationscited by 36
- 2021Approximation capability of two hidden layer feedforward neural networks with fixed weightscited by 24
- 2015A single hidden layer feedforward network with only one neuron in the hidden layer can approximate any univariate functioncited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024On Expressive Power of Quantized Neural Networks under Fixed-Point Arithmetic
- 2021Approximation of functions with one-bit neural networks
- 2022Neural Network Verification as Piecewise Linear Optimization: Formulations for the Composition of Staircase Functions
- 2024Three Quantization Regimes for ReLU Networks
- 2024On the Complexity of Neural Computation in Superposition
- 2025Compressed Computation: Dense Circuits in a Toy Model of the Universal-AND Problem
- 2021Universal Approximation Theorems of Fully Connected Binarized Neural Networks
- 2023On the Optimal Expressive Power of ReLU DNNs and Its Application in Approximation with Kolmogorov Superposition Theorem