# Approximation Rates for Neural Networks with General Activation Functions

arXiv: https://arxiv.org/abs/1904.02311 · HTML: https://mlsanity.com/paper/1904.02311/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Apr 2019.

## Built on by (most cited first, 3 of 3)

- [Optimal bump functions for shallow ReLU networks: Weight decay, depth separation and the curse of dimensionality](https://mlsanity.com/paper/2209.01173/) (2022)
- [Qualitative neural network approximation over R and C: Elementary proofs for analytic and polynomial activation](https://mlsanity.com/paper/2203.13410/) (2022)
- [Elliptic Regularity Theory in Barron Spaces and Applications to the Deep Ritz Method](https://mlsanity.com/paper/2607.25100/) (2026)

## Abstract (arXiv)

We prove some new results concerning the approximation rate of neural networks with general activation functions. Our first result concerns the rate of approximation of a two layer neural network with a polynomially-decaying non-sigmoidal activation function. We extend the dimension independent approximation rates previously obtained to this new class of activation functions. Our second result gives a weaker, but still dimension independent, approximation rate for a larger class of activation functions, removing the polynomial decay assumption. This result applies to any bounded, integrable activation function. Finally, we show that a stratified sampling approach can be used to improve the approximation rate for polynomially decaying activation functions under mild additional assumptions.