mlsanityData to 14 Oct 2026

Qualitative neural network approximation over R and C: Elementary proofs for analytic and polynomial activation

arXiv 2203.13410 ↗First posted 25 Mar 2022cs.LG math.FA stat.MLcited by 2

Not on the map. Why?

Abstract (arXiv)

In this article, we prove approximation theorems in classes of deep and shallow neural networks with analytic activation functions by elementary arguments. We prove for both real and complex networks with non-polynomial activation that the closure of the class of neural networks coincides with the closure of the space of polynomials. The closure can further be characterized by the Stone-Weierstrass theorem (in the real case) and Mergelyan's theorem (in the complex case). In the real case, we further prove approximation results for networks with higher-dimensional harmonic activation and orthogonally projected linear maps. We further show that fully connected and residual networks of large depth with polynomial activation functions can approximate any polynomial under certain width requirements. All proofs are entirely elementary.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 36 of its 85 references that are in the map. First posted on arXiv 25 Mar 2022.

Builds on

36 of 85 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018Neural Ordinary Differential Equationscited by 3,779
  3. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  4. 2016Error bounds for approximations with deep ReLU networkscited by 574
  5. 2015An Introduction to Convolutional Neural Networkscited by 514
  6. 2018Hyperbolic Neural Networkscited by 402
  7. 2019Hyperbolic Graph Convolutional Neural Networkscited by 402
  8. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 331
  9. 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 257
  10. 2019Hyperbolic Graph Neural Networkscited by 244
  11. 2017Deep Complex Networkscited by 217
  12. 2017Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanicscited by 214
24 more
  1. 2015The Power of Depth for Feedforward Neural Networkscited by 183
  2. 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 179
  3. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 162
  4. 2021Hyperbolic Deep Neural Networks: A Surveycited by 139
  5. 2017Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activationscited by 104
  6. 2019The phase diagram of approximation rates for deep neural networkscited by 88
  7. 2016Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with ℓ1 and ℓ0 Controlscited by 84
  8. 2017Maximum Principle Based Algorithms for Deep Learningcited by 78
  9. 2018A Priori Estimates of the Population Risk for Two-layer Neural Networkscited by 76
  10. 2019Deep ReLU network approximation of functions on a manifoldcited by 74
  11. 2014DenseNet: Implementing Efficient ConvNet Descriptor Pyramidscited by 66
  12. 2016Deep vs. shallow networks : An approximation theory perspectivecited by 63
  13. 2020Representation formulas and pointwise properties for Barron functionscited by 62
  14. 2021What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theorycited by 60
  15. 2019Approximation spaces of deep neural networkscited by 51
  16. 2018Exponential Convergence of the Deep Neural Network Approximation for Analytic Functionscited by 38
  17. 2018Deep, Skinny Neural Networks are not Universal Approximatorscited by 38
  18. 2017Better than Real: Complex-valued Neural Nets for MRI Fingerprintingcited by 30
  19. 2020The universal approximation theorem for complex-valued neural networkscited by 28
  20. 2020Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernelscited by 19
  21. 2020DeePKS: a comprehensive data-driven approach towards chemically accurate density functional theorycited by 15
  22. 2021Neural Network Approximation of Refinable Functionscited by 15
  23. 2015A mathematical motivation for complex-valued convolutional networkscited by 13
  24. 2019Approximation Rates for Neural Networks with General Activation Functionscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 0; 2026: 1.

2 papers.

  1. 2023Universal approximation with complex-valued deep narrow neural networkscited by 3
  2. 2026A holomorphic neural network framework for 3D boundary value problems governed by harmonic potentialscited by 2