# Approximation of Smoothness Classes by Deep Rectifier Networks

arXiv: https://arxiv.org/abs/2007.15645 · HTML: https://mlsanity.com/paper/2007.15645/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jul 2020.

## Built on by (most cited first, 6 of 6)

- [Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations](https://mlsanity.com/paper/2206.09527/) (2022)
- [Differentiable Neural Networks with RePU Activation: with Applications to Score Estimation and Isotonic Regression](https://mlsanity.com/paper/2305.00608/) (2023)
- [Theoretical Exploration of Solutions of Feedforward ReLU Networks](https://mlsanity.com/paper/2202.01919/) (2022)
- [Approximation Theory of Tree Tensor Networks: Tensorized Multivariate Functions](https://mlsanity.com/paper/2101.11932/) (2021)
- [Theoretical Insights into CycleGAN: Analyzing Approximation and Estimation Errors in Unpaired Data Generation](https://mlsanity.com/paper/2407.11678/) (2024)
- [ReLU Networks for Model Predictive Control: Network Complexity and Performance Guarantees](https://mlsanity.com/paper/2601.16764/) (2026)

## Abstract (arXiv)

We consider approximation rates of sparsely connected deep rectified linear unit (ReLU) and rectified power unit (RePU) neural networks for functions in Besov spaces Bαq(Lp) in arbitrary dimension d, on general domains. We show that alertdeep rectifier networks with a fixed activation function attain optimal or near to optimal approximation rates for functions in the Besov space Bατ(Lτ) on the critical embedding line 1/τ=α/d+1/p for arbitrary smoothness order α>0. Using interpolation theory, this implies that the entire range of smoothness classes at or above the critical line is (near to) optimally approximated by deep ReLU/RePU networks.