# Approximation capability of two hidden layer feedforward neural networks with fixed weights

arXiv: https://arxiv.org/abs/2101.09181 · HTML: https://mlsanity.com/paper/2101.09181/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jan 2021.

## Built on by (most cited first, 24 of 24)

- [On the approximation of functions by tanh neural networks](https://mlsanity.com/paper/2104.08938/) (2021)
- [On the Kolmogorov neural networks](https://mlsanity.com/paper/2311.00049/) (2023)
- [The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimation](https://mlsanity.com/paper/2103.11488/) (2021)
- [Addressing common misinterpretations of KART and UAT in neural network literature](https://mlsanity.com/paper/2408.16389/) (2024)
- [KASAM: Spline Additive Models for Function Approximation](https://mlsanity.com/paper/2205.06376/) (2022)
- [Robust Weight Initialization for Tanh Neural Networks with Fixed Point Analysis](https://mlsanity.com/paper/2410.02242/) (2024)
- [Universal approximation theorem for neural networks with inputs from a topological vector space](https://mlsanity.com/paper/2409.12913/) (2024)
- [Are Two Hidden Layers Still Enough for the Physics-Informed Neural Networks?](https://mlsanity.com/paper/2412.19235/) (2024)
- [Approximation error of single hidden layer neural networks with fixed weights](https://mlsanity.com/paper/2202.03289/) (2022)
- [Approximation properties of neural ODEs](https://mlsanity.com/paper/2503.15696/) (2025)
- [On shallow feedforward neural networks with inputs from a topological space](https://mlsanity.com/paper/2504.02321/) (2025)
- [SGD method for entropy error function with smoothing l0 regularization for neural networks](https://mlsanity.com/paper/2405.18552/) (2024)
- [Numerical Approximation Capacity of Neural Networks with Bounded Parameters: Do Limits Exist, and How Can They Be Measured?](https://mlsanity.com/paper/2409.16697/) (2024)
- [Expressivity of Quadratic Neural ODEs](https://mlsanity.com/paper/2504.09385/) (2025)
- [On Explicit Super-Expressive Approximation for Neural Networks](https://mlsanity.com/paper/2607.06781/) (2026)
- [Efficient Computation of Confidence Sets Using Classification on Equidistributed Grids](https://mlsanity.com/paper/2401.01804/) (2024)
- [Learnability of high-dimensional targets by two-parameter models and gradient flow](https://mlsanity.com/paper/2402.17089/) (2024)
- [Fingerprinting web servers through Transformer-encoded HTTP response headers](https://mlsanity.com/paper/2404.00056/) (2024)
- [Universal Approximation Theorem for Input-Connected Multilayer Perceptrons](https://mlsanity.com/paper/2601.14026/) (2026)
- [Universality of shallow and deep neural networks on non-Euclidean spaces](https://mlsanity.com/paper/2602.23381/) (2026)
- [Man, Machine, and Mathematics](https://mlsanity.com/paper/2604.27052/) (2026)
- [StreamPhy: Streaming Inference of High-Dimensional Physical Dynamics via State Space Models](https://mlsanity.com/paper/2605.07384/) (2026)
- [Solving In-Table Prediction Problems by Deep Neural Networks with Performance Evaluation Using Synthetic Data](https://mlsanity.com/paper/2609.01262/) (2026)
- [Arbitrary-Accuracy Neural Approximation with Optimal Neuron Count and Near-Optimal Bit Complexity](https://mlsanity.com/paper/2609.35628/) (2026)

## Abstract (arXiv)

We algorithmically construct a two hidden layer feedforward neural network (TLFN) model with the weights fixed as the unit coordinate vectors of the d-dimensional Euclidean space and having 3d+2 number of hidden neurons in total, which can approximate any continuous d-variable function with an arbitrary precision. This result, in particular, shows an advantage of the TLFN model over the single hidden layer feedforward neural network (SLFN) model, since SLFNs with fixed weights do not have the capability of approximating multivariate functions.