# Optimal Approximation Complexity of High-Dimensional Functions with Neural Networks

arXiv: https://arxiv.org/abs/2301.13091 · HTML: https://mlsanity.com/paper/2301.13091/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 17 references that are in the map.
- First posted on arXiv 30 Jan 2023.

## Builds on (most cited first, 5 of 5)

- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Countering Adversarial Images using Input Transformations](https://mlsanity.com/paper/1711.00117/) (2017)
- [Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Review](https://mlsanity.com/paper/1611.00740/) (2016)
- [Representation Benefits of Deep Feedforward Networks](https://mlsanity.com/paper/1509.08101/) (2015)
- [Rational neural networks](https://mlsanity.com/paper/2004.01902/) (2020)

## Abstract (arXiv)

We investigate properties of neural networks that use both ReLU and x2 as activation functions and build upon previous results to show that both analytic functions and functions in Sobolev spaces can be approximated by such networks of constant depth to arbitrary accuracy, demonstrating optimal order approximation rates across all nonlinear approximators, including standard ReLU networks. We then show how to leverage low local dimensionality in some contexts to overcome the curse of dimensionality, obtaining approximation rates that are optimal for unknown lower-dimensional subspaces.