mlsanityData to 14 Oct 2026

Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations

arXiv 2206.09527 ↗First posted 20 Jun 2022math.NA cs.NA math.STcited by 29

Not on the map. Why?

Abstract (arXiv)

This paper investigates the approximation properties of deep neural networks with piecewise-polynomial activation functions. We derive the required depth, width, and sparsity of a deep neural network to approximate any H"older smooth function up to a given approximation error in H"older norms in such a way that all weights of this neural network are bounded by 1. The latter feature is essential to control generalization errors in many statistical and machine learning applications.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 22 of its 43 references that are in the map. First posted on arXiv 20 Jun 2022.

Builds on

22 of 43 references.

References · 22

  1. 2016Error bounds for approximations with deep ReLU networkscited by 574
  2. 2017Solving high-dimensional partial differential equations using deep learningcited by 556
  3. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 433
  4. 2019Universal Approximation with Deep Narrow Networkscited by 200
  5. 2020Deep Network Approximation for Smooth Functionscited by 164
  6. 2019Deep Network Approximation Characterized by Number of Neuronscited by 122
  7. 2017Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activationscited by 104
  8. 2019Error bounds for approximations with deep ReLU neural networks in Ws,p normscited by 95
  9. 2020Minimum Width for Universal Approximationcited by 94
  10. 2019The phase diagram of approximation rates for deep neural networkscited by 88
  11. 2018Optimal Control Via Neural Networks: A Convex Approachcited by 85
  12. 2016Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with ℓ1 and ℓ0 Controlscited by 84
10 more
  1. 2021On the approximation of functions by tanh neural networkscited by 81
  2. 2020Neural Network Approximation: Three Hidden Layers Are Enoughcited by 63
  3. 2019Approximation spaces of deep neural networkscited by 51
  4. 2020Numerical Solution of the Parametric Diffusion Equation by Deep Neural Networkscited by 27
  5. 2020Approximating smooth functions by deep neural networks with sigmoid activation functioncited by 18
  6. 2021A Neural Network Approach for High-Dimensional Optimal Control Applied to Multi-Agent Path Findingcited by 18
  7. 2020Approximations with deep neural networks in Sobolev time-spacecited by 13
  8. 2019PowerNet: Efficient Representations of Polynomials and Smooth Functions by Deep Neural Networks with Rectified Power Unitscited by 12
  9. 2020Approximation of Smoothness Classes by Deep Rectifier Networkscited by 6
  10. 2021Evolved neuromorphic radar-based altitude controller for an autonomous open-source blimpcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 7; 2025: 11; 2026: 7.

29 papers.

Sort:

By area:

  1. 2024Enhancing Accuracy in Generative Models via Knowledge Transfercited by 7
  2. 2023A prediction and behavioural analysis of machine learning methods for modelling travel mode choicecited by 4
  3. 2023Theoretical guarantees for neural control variates in MCMCcited by 4
  4. 2025Approximation Rates in Besov Norms and Sample-Complexity of Kolmogorov-Arnold Networks with Residual Connectionscited by 4
  5. 2024A Wasserstein perspective of Vanilla GANscited by 3
  6. 2024On the estimation rate of Bayesian PINN for inverse problemscited by 3
  7. 2025Numerical and statistical analysis of NeuralODE with Runge-Kutta time integrationcited by 3
  8. 2025Learning Brenier Potentials with Convex Generative Adversarial Neural Networkscited by 3
  9. 2025Simultaneous Approximation of the Score Function and Its Derivatives by Deep Neural Networkscited by 3
  10. 2024Convergence of the Deep Galerkin Method for Finite State Mean Field Control Problemscited by 2
  11. 2025C1-approximation with rational functions and rational neural networkscited by 2
  12. 2025Expressive Power of Deep Networks on Manifolds: Simultaneous Approximationcited by 2
17 more
  1. 2025Approximation Capabilities of Feedforward Neural Networks with GELU Activationscited by 2
  2. 2023Learning Green's Function Efficiently Using Low-Rank Approximationscited by 1
  3. 2024Universality of reservoir systems with recurrent neural networkscited by 1
  4. 2025Improving Mixup Calibration with Wasserstein Distributionally Robust Optimizationcited by 1
  5. 2025Stackelberg Coupling of Online Representation Learning and Reinforcement Learningcited by 1
  6. 2025Physically consistent model learning for reaction-diffusion systemscited by 1
  7. 2026A Quantitative Approximation Framework for Flow Distillation in Diffusion Modelscited by 1
  8. 2023A statistical perspective on algorithm unrolling models for inverse problemscited by 0
  9. 2024On the growth of the parameters of approximating ReLU neural networkscited by 0
  10. 2024Scaling-based Data Augmentation for Generative Models and its Theoretical Extensioncited by 0
  11. 2025Consistency of Learned Sparse Grid Quadrature Rules using NeuralODEscited by 0
  12. 2026Regularity of Solutions to Beckmann's Parametric Optimal Transportcited by 0
  13. 2026Generative Adversarial Learning from Deterministic Processescited by 0
  14. 2026Mitigating the Curse of Dimensionality in Uniform Convergence of Deep Neural Networks via Smooth Activationscited by 0
  15. 2026Nonparametric inference for density-dependent McKean--Vlasov diffusionscited by 0
  16. 2026Deep Barycentric Regression for Optimal Transport Map Estimation and its Statistical Optimalitycited by 0
  17. 2026Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networkscited by 0