# Arbitrary-Accuracy Neural Approximation with Optimal Neuron Count and Near-Optimal Bit Complexity

arXiv: https://arxiv.org/abs/2609.35628 · HTML: https://mlsanity.com/paper/2609.35628/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 19 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Builds on (most cited first, 9 of 9)

- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Optimal approximation of piecewise smooth functions using deep ReLU neural networks](https://mlsanity.com/paper/1709.05289/) (2017)
- [Optimal approximation of continuous functions by very deep ReLU networks](https://mlsanity.com/paper/1802.03620/) (2018)
- [Optimal Approximation Rate of ReLU Networks in terms of Width and Depth](https://mlsanity.com/paper/2103.00502/) (2021)
- [Neural Network Approximation: Three Hidden Layers Are Enough](https://mlsanity.com/paper/2010.14075/) (2020)
- [Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neurons](https://mlsanity.com/paper/2107.02397/) (2021)
- [Elementary superexpressive activations](https://mlsanity.com/paper/2102.10911/) (2021)
- [Approximation capability of two hidden layer feedforward neural networks with fixed weights](https://mlsanity.com/paper/2101.09181/) (2021)
- [A single hidden layer feedforward network with only one neuron in the hidden layer can approximate any univariate function](https://mlsanity.com/paper/1601.00013/) (2015)

## Abstract (arXiv)

We study the minimum number of hidden neurons required for arbitrary-accuracy approximation of multivariate H"older-continuous functions on [0,1]d and the associated encoding complexity. For d≥ 2, we construct a fixed, explicitly defined activation function for which a closed-form network with two hidden layers of widths d and 1 achieves arbitrary accuracy in the uniform norm. We prove that d+1 is the exact minimum total number of hidden neurons among standard feedforward networks with locally integrable activations and affine outputs. We further give a simpler construction using a single elementary activation that combines the floor and exponential functions. This construction requires three hidden layers of widths d, 1, and 2, only two neurons above the minimum. If a skip connection is allowed, widths d, 1, and 1 suffice. These constructions use explicit grid addressing and integer encoding of quantized function values. For a bounded α-H"older class, they require O(varepsilon-d/αlog(1/varepsilon)) bits, matching the metric-entropy lower bound up to a logarithmic factor.