# Approximation with Random Shallow ReLU Networks with Applications to Model Reference Adaptive Control

arXiv: https://arxiv.org/abs/2403.17142 · HTML: https://mlsanity.com/paper/2403.17142/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 25 Mar 2024.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 13 of 13)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [A Convergence Theory for Deep Learning via Over-Parameterization](https://mlsanity.com/paper/1811.03962/) (2018)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Gradient Descent Finds Global Minima of Deep Neural Networks](https://mlsanity.com/paper/1811.03804/) (2018)
- [On the Power and Limitations of Random Features for Understanding Neural Networks](https://mlsanity.com/paper/1904.00687/) (2019)
- [Neural Network with Unbounded Activation Functions is Universal Approximator](https://mlsanity.com/paper/1505.03654/) (2015)
- [On the Approximation Power of Two-Layer Networks of Random ReLUs](https://mlsanity.com/paper/2102.02336/) (2021)
- [Safe Exploration in Model-based Reinforcement Learning using Control Barrier Functions](https://mlsanity.com/paper/2104.08171/) (2021)
- [Neural network integral representations with the ReLU activation function](https://mlsanity.com/paper/1910.02743/) (2019)
- [Non-Asymptotic Pointwise and Worst-Case Bounds for Classical Spectrum Estimators](https://mlsanity.com/paper/2303.11908/) (2023)
- [How Powerful are Shallow Neural Networks with Bandlimited Random Weights?](https://mlsanity.com/paper/2008.08427/) (2020)
- [Sufficient Conditions for Persistency of Excitation with Step and ReLU Activation Functions](https://mlsanity.com/paper/2209.06286/) (2022)
- [Robust Model Based Reinforcement Learning Using L1 Adaptive Control](https://mlsanity.com/paper/2403.14860/) (2024)

## Built on by (most cited first, 4 of 4)

- [Orthogonal greedy algorithm for linear operator learning with shallow neural network](https://mlsanity.com/paper/2501.02791/) (2025)
- [Function Gradient Approximation with Random Shallow ReLU Networks with Control Applications](https://mlsanity.com/paper/2410.05071/) (2024)
- [A Neural Network Algorithm for KL Divergence Estimation with Quantitative Error Bounds](https://mlsanity.com/paper/2510.05386/) (2025)
- [Asynchronous Cooperative Online Learning for Multi-Robot Control under Computational Delays](https://mlsanity.com/paper/2608.29562/) (2026)

## Abstract (arXiv)

Neural networks are regularly employed in adaptive control of nonlinear systems and related methods of reinforcement learning. A common architecture uses a neural network with a single hidden layer (i.e. a shallow network), in which the weights and biases are fixed in advance and only the output layer is trained. While classical results show that there exist neural networks of this type that can approximate arbitrary continuous functions over bounded regions, they are non-constructive, and the networks used in practice have no approximation guarantees. Thus, the approximation properties required for control with neural networks are assumed, rather than proved. In this paper, we aim to fill this gap by showing that for sufficiently smooth functions, ReLU networks with randomly generated weights and biases achieve L∞ error of O(m-1/2) with high probability, where m is the number of neurons. It suffices to generate the weights uniformly over a sphere and the biases uniformly over an interval. We show how the result can be used to get approximations of required accuracy in a model reference adaptive control application.