# Random Projection Neural Networks of Best Approximation: Convergence theory and practical applications

arXiv: https://arxiv.org/abs/2402.11397 · HTML: https://mlsanity.com/paper/2402.11397/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 85 references that are in the map.
- First posted on arXiv 17 Feb 2024.

## Builds on (most cited first, 8 of 8)

- [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021)
- [The Barron Space and the Flow-induced Function Spaces for Neural Network Models](https://mlsanity.com/paper/1906.08039/) (2019)
- [Exponential Convergence of the Deep Neural Network Approximation for Analytic Functions](https://mlsanity.com/paper/1807.00297/) (2018)
- [On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elements](https://mlsanity.com/paper/2110.14121/) (2021)
- [Sampling weights of deep neural networks](https://mlsanity.com/paper/2306.16830/) (2023)
- [Learning normal form autoencoders for data-driven discovery of universal,parameter-dependent governing equations](https://mlsanity.com/paper/2106.05102/) (2021)
- [Slow Invariant Manifolds of Singularly Perturbed Systems via Physics-Informed Machine Learning](https://mlsanity.com/paper/2309.07946/) (2023)
- [Tasks Makyth Models: Machine Learning Assisted Surrogates for Tipping Points](https://mlsanity.com/paper/2309.14334/) (2023)

## Built on by (most cited first, 11 of 11)

- [RandONet: Shallow-Networks with Random Projections for learning linear and nonlinear operators](https://mlsanity.com/paper/2406.05470/) (2024)
- [Enabling Local Neural Operators to perform Equation-Free System-Level Analysis](https://mlsanity.com/paper/2505.02308/) (2025)
- [Rapid training of Hamiltonian graph networks using random features](https://mlsanity.com/paper/2506.06558/) (2025)
- [Equation-Free Coarse Control of Distributed Parameter Systems via Local Neural Operators](https://mlsanity.com/paper/2509.23975/) (2025)
- [HEATNETs: Explainable Random Feature Neural Networks for High-Dimensional Parabolic PDEs](https://mlsanity.com/paper/2511.00886/) (2025)
- [Linear Stability Analysis of Physics-Informed Random Projection Neural Networks for ODEs](https://mlsanity.com/paper/2408.15393/) (2024)
- [Improving statistical learning methods via features selection without replacement sampling and random projection](https://mlsanity.com/paper/2506.00053/) (2025)
- [Kantorovich-Type Stochastic Neural Network Operators for the Mean-Square Approximation of Certain Second-Order Stochastic Processes](https://mlsanity.com/paper/2601.03634/) (2026)
- [RANDSMAPs: Random-Feature/multi-Scale Neural Decoders with Mass Preservation](https://mlsanity.com/paper/2601.14794/) (2026)
- [Stability and Bifurcation Analysis of Nonlinear PDEs via Random Projection-based PINNs: A Krylov-Arnoldi Approach](https://mlsanity.com/paper/2603.21568/) (2026)
- [Spectral Convergence of Random Feature Method in Multiple Dimensions](https://mlsanity.com/paper/2609.03401/) (2026)

## Abstract (arXiv)

We investigate the concept of Best Approximation for Feedforward Neural Networks (FNN) and explore their convergence properties through the lens of Random Projection (RPNNs). RPNNs have predetermined and fixed, once and for all, internal weights and biases, offering computational efficiency. We demonstrate that there exists a choice of external weights, for any family of such RPNNs, with non-polynomial infinitely differentiable activation functions, that exhibit an exponential convergence rate when approximating any infinitely differentiable function. For illustration purposes, we test the proposed RPNN-based function approximation, with parsimoniously chosen basis functions, across five benchmark function approximation problems. Results show that RPNNs achieve comparable performance to established methods such as Legendre Polynomials, highlighting their potential for efficient and accurate function approximation.