mlsanityData to 7 Oct 2026

Deep ReLU networks and high-order finite element methods II: Chebyshev emulation

arXiv 2310.07261 ↗First posted 11 Oct 2023math.NA cs.LG cs.NAcited by 3

Not on the map. Why?

Abstract (arXiv)

We show expression rates and stability in Sobolev norms of deep feedforward ReLU neural networks (NNs) in terms of the number of parameters defining the NN for continuous, piecewise polynomial functions, on arbitrary, finite partitions T of a bounded interval (a,b). Novel constructions of ReLU NN surrogates encoding function approximations in terms of Chebyshev polynomial expansion coefficients are developed which require fewer neurons than previous constructions. Chebyshev coefficients can be computed easily from the values of the function in the Clenshaw--Curtis points using the inverse fast Fourier transform. Bounds on expression rates and stability are obtained that are superior to those of constructions based on ReLU NN emulations of monomials as considered in [Opschoor, Petersen and Schwab, 2020] and [Montanelli, Yang and Du, 2021]. All emulation bounds are explicit in terms of the (arbitrary) partition of the interval, the target emulation accuracy and the polynomial degree in each element of the partition. ReLU NN emulation error estimates are provided for various classes of functions and norms, commonly encountered in numerical analysis. In particular, we show exponential ReLU emulation rate bounds for analytic functions with point singularities and develop an interface between Chebfun approximations and constructive ReLU NN emulations.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 18 of its 48 references that are in the map. First posted on arXiv 11 Oct 2023.

Builds on

18 of 48 references.

References · 18

  1. 2016Error bounds for approximations with deep ReLU networkscited by 567
  2. 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 176
  3. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 160
  4. 2019Deep Network Approximation Characterized by Number of Neuronscited by 120
  5. 2019Error bounds for approximations with deep ReLU neural networks in Ws,p normscited by 95
  6. 2019The phase diagram of approximation rates for deep neural networkscited by 86
  7. 2021On the approximation of functions by tanh neural networkscited by 81
  8. 2020Neural Network Approximation: Three Hidden Layers Are Enoughcited by 63
  9. 2019Nonlinear Approximation and (Deep) ReLU Networkscited by 45
  10. 2021Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neuronscited by 41
  11. 2021Elementary superexpressive activationscited by 35
  12. 2020Exponential ReLU Neural Network Approximation Rates for Point and Edge Singularitiescited by 20
6 more
  1. 2018A Framework for the construction of upper bounds on the number of affine linear regions of ReLU feed-forward neural networkscited by 15
  2. 2022An Exact Mapping From ReLU Networks to Spiking Neural Networkscited by 12
  3. 2018The universal approximation power of finite-width deep ReLU networkscited by 11
  4. 2021Optimal learning of high-dimensional classification problems using deep neural networkscited by 9
  5. 2022De Rham compatible Deep Neural Network FEMcited by 4
  6. 2023Expressivity of Spiking Neural Networkscited by 4

Built on by

Cited by year

Citing papers per year: 2024: 3.

3 papers.

  1. 2024Exponential Expressivity of ReLUk Neural Networks on Gevrey Classes with Point Singularitiescited by 3
  2. 2024Neural Networks for Singular Perturbationscited by 1
  3. 2024Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximationcited by 1