# Neural Network Approximation of Refinable Functions

arXiv: https://arxiv.org/abs/2107.13191 · HTML: https://mlsanity.com/paper/2107.13191/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 29 references that are in the map.
- First posted on arXiv 28 Jul 2021.

## Builds on (most cited first, 9 of 9)

- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [On the Expressive Power of Deep Neural Networks](https://mlsanity.com/paper/1606.05336/) (2016)
- [Benefits of depth in neural networks](https://mlsanity.com/paper/1602.04485/) (2016)
- [The Power of Depth for Feedforward Neural Networks](https://mlsanity.com/paper/1512.03965/) (2015)
- [Deep Neural Network Approximation Theory](https://mlsanity.com/paper/1901.02220/) (2019)
- [Representation Benefits of Deep Feedforward Networks](https://mlsanity.com/paper/1509.08101/) (2015)
- [Nonlinear Approximation and (Deep) ReLU Networks](https://mlsanity.com/paper/1905.02199/) (2019)
- [Exponential Convergence of the Deep Neural Network Approximation for Analytic Functions](https://mlsanity.com/paper/1807.00297/) (2018)
- [Expression of Fractals Through Neural Network Functions](https://mlsanity.com/paper/1905.11345/) (2019)

## Built on by (most cited first, 15 of 15)

- [Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spaces](https://mlsanity.com/paper/2211.14400/) (2022)
- [Polarity Sampling: Quality and Diversity Control of Pre-Trained Generative Networks via Singular Values](https://mlsanity.com/paper/2203.01993/) (2022)
- [Singular Value Perturbation and Deep Network Optimization](https://mlsanity.com/paper/2203.03099/) (2022)
- [On Minimal Depth in Neural Networks](https://mlsanity.com/paper/2402.15315/) (2024)
- [Exact Loop Controllers for ReLU Realization of Homogeneous Curve Refinements](https://mlsanity.com/paper/2605.01655/) (2026)
- [Qualitative neural network approximation over R and C: Elementary proofs for analytic and polynomial activation](https://mlsanity.com/paper/2203.13410/) (2022)
- [Do stable neural networks exist for classification problems? -- A new view on stability in AI](https://mlsanity.com/paper/2401.07874/) (2024)
- [Sobolev-type embeddings for neural network approximation spaces](https://mlsanity.com/paper/2110.15304/) (2021)
- [Mitigating over-exploration in latent space optimization using LES](https://mlsanity.com/paper/2406.09657/) (2024)
- [Exact ReLU realization of affine one-dimensional refinement iterates via residual memory and offset frames](https://mlsanity.com/paper/2607.20586/) (2026)
- [On the algorithmic construction of deep ReLU networks](https://mlsanity.com/paper/2506.19104/) (2025)
- [Neural Network Approximation: A View from Polytope Decomposition](https://mlsanity.com/paper/2601.18264/) (2026)
- [Exact ReLU realization of tensor-product refinement iterates](https://mlsanity.com/paper/2605.03917/) (2026)
- [Neural network realization of binary refinement iterates via a two-chart atlas selector](https://mlsanity.com/paper/2608.02624/) (2026)
- [Exact ReLU realization of binary affine refinement iterates via reflection folding and cone switching](https://mlsanity.com/paper/2609.11962/) (2026)

## Abstract (arXiv)

In the desire to quantify the success of neural networks in deep learning and other applications, there is a great interest in understanding which functions are efficiently approximated by the outputs of neural networks. By now, there exists a variety of results which show that a wide range of functions can be approximated with sometimes surprising accuracy by these outputs. For example, it is known that the set of functions that can be approximated with exponential accuracy (in terms of the number of parameters used) includes, on one hand, very smooth functions such as polynomials and analytic functions (see e.g. citeE,S,Y) and, on the other hand, very rough functions such as the Weierstrass function (see e.g. citeEPGB,DDFHP), which is nowhere differentiable. In this paper, we add to the latter class of rough functions by showing that it also includes refinable functions. Namely, we show that refinable functions are approximated by the outputs of deep ReLU networks with a fixed width and increasing depth with accuracy exponential in terms of their number of parameters. Our results apply to functions used in the standard construction of wavelets as well as to functions constructed via subdivision algorithms in Computer Aided Geometric Design.