# On Explicit Super-Expressive Approximation for Neural Networks

arXiv: https://arxiv.org/abs/2607.06781 · HTML: https://mlsanity.com/paper/2607.06781/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 35 references that are in the map.
- First posted on arXiv 7 Jul 2026.

## Builds on (most cited first, 15 of 15)

- [Are Transformers universal approximators of sequence-to-sequence functions?](https://mlsanity.com/paper/1912.10077/) (2019)
- [Universality of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1805.10769/) (2018)
- [Deep Network Approximation for Smooth Functions](https://mlsanity.com/paper/2001.03040/) (2020)
- [Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations](https://mlsanity.com/paper/1811.01558/) (2018)
- [Deep Learning via Dynamical Systems: An Approximation Perspective](https://mlsanity.com/paper/1912.10382/) (2019)
- [Rational neural networks](https://mlsanity.com/paper/2004.01902/) (2020)
- [Neural Network Approximation: Three Hidden Layers Are Enough](https://mlsanity.com/paper/2010.14075/) (2020)
- [Approximation capability of two hidden layer feedforward neural networks with fixed weights](https://mlsanity.com/paper/2101.09181/) (2021)
- [On the approximation by single hidden layer feedforward neural networks with fixed weights](https://mlsanity.com/paper/1708.06219/) (2017)
- [PowerNet: Efficient Representations of Polynomials and Smooth Functions by Deep Neural Networks with Rectified Power Units](https://mlsanity.com/paper/1909.05136/) (2019)
- [Differentiable Neural Networks with RePU Activation: with Applications to Score Estimation and Isotonic Regression](https://mlsanity.com/paper/2305.00608/) (2023)
- [Don't Fear Peculiar Activation Functions: EUAF and Beyond](https://mlsanity.com/paper/2407.09580/) (2024)
- [A single hidden layer feedforward network with only one neuron in the hidden layer can approximate any univariate function](https://mlsanity.com/paper/1601.00013/) (2015)
- [Data Optimization in Deep Learning: A Survey](https://mlsanity.com/paper/2310.16499/) (2023)
- [Neural networks with superexpressive activations and integer weights](https://mlsanity.com/paper/2105.09917/) (2021)

## Built on by (most cited first, 1 of 1)

- [Optimal Tradeoffs Between Network Size and Parameter Magnitude in Neural Approximation and Minimax Regression](https://mlsanity.com/paper/2609.25710/) (2026)

## Abstract (arXiv)

In this work, we investigate the fixed-architecture neural network approximation with explicit parameter bounds and elementary activations. While prior work demonstrated super-expressive approximation using fixed-size networks, they lack quantitative and non-asymptotic characterizations of parameter magnitude with respect to the approximation error. We resolve this issue by introducing the Chinese Remainder Theorem as a constructive encoding mechanism. For Lipschitz continuous functions on [0,1]D, we construct a width-maxD,4, depth-5 network with explicit parameter-error trade-offs. For H"older-smooth functions in Cr,γAleft([0,1]Dright), our fixed network of width max2D, D+5N+1 and depth r + 9 achieves the parameter magnitude P bounded by log2 P=Obigl(varepsilon-2D/(r+γ)log(1/varepsilon)bigr). This is the dual result compared to those in the parameter-bounded and architecture-unbounded paradigm.