# Approximation of Nonlinear Functionals Using Deep ReLU Networks

arXiv: https://arxiv.org/abs/2304.04443 · HTML: https://mlsanity.com/paper/2304.04443/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 30 references that are in the map.
- First posted on arXiv 10 Apr 2023.

## Builds on (most cited first, 7 of 7)

- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Benefits of depth in neural networks](https://mlsanity.com/paper/1602.04485/) (2016)
- [Optimal approximation of piecewise smooth functions using deep ReLU neural networks](https://mlsanity.com/paper/1709.05289/) (2017)
- [Universality of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1805.10769/) (2018)
- [Optimal approximation of continuous functions by very deep ReLU networks](https://mlsanity.com/paper/1802.03620/) (2018)
- [Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with  ℓ1  and  ℓ0  Controls](https://mlsanity.com/paper/1607.07819/) (2016)
- [Provable approximation properties for deep neural networks](https://mlsanity.com/paper/1509.07385/) (2015)

## Built on by (most cited first, 8 of 8)

- [Expressivity and Approximation Properties of Deep Neural Networks with ReLUk Activation](https://mlsanity.com/paper/2312.16483/) (2023)
- [Nonlinear functional regression by functional deep neural network with kernel embedding](https://mlsanity.com/paper/2401.02890/) (2024)
- [Spherical Analysis of Learning Nonlinear Functionals](https://mlsanity.com/paper/2410.01047/) (2024)
- [Data Topology-Dependent Upper Bounds of Neural Network Widths](https://mlsanity.com/paper/2305.16375/) (2023)
- [A Novel Neural Filter to Improve Accuracy of Neural Network Models of Dynamic Systems](https://mlsanity.com/paper/2409.13654/) (2024)
- [Sparse-Aware Neural Networks for Nonlinear Functionals: Mitigating the Exponential Dependence on Dimension](https://mlsanity.com/paper/2604.06774/) (2026)
- [Generalization Analysis for Classification on Korobov Space](https://mlsanity.com/paper/2509.22748/) (2025)
- [Generalization Analysis of Transformers in Distribution Regression](https://mlsanity.com/paper/2606.29256/) (2026)

## Abstract (arXiv)

In recent years, functional neural networks have been proposed and studied in order to approximate nonlinear continuous functionals defined on Lp([-1, 1]s) for integers s≥1 and 1≤ p<∞. However, their theoretical properties are largely unknown beyond universality of approximation or the existing analysis does not apply to the rectified linear unit (ReLU) activation function. To fill in this void, we investigate here the approximation power of functional deep neural networks associated with the ReLU activation function by constructing a continuous piecewise linear interpolation under a simple triangulation. In addition, we establish rates of approximation of the proposed functional deep ReLU networks under mild regularity conditions. Finally, our study may also shed some light on the understanding of functional data learning algorithms.