mlsanityData to 14 Oct 2026

Approximation and interpolation of deep neural networks

arXiv 2304.10552 ↗First posted 20 Apr 2023cs.LG math.OC math.PRcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper, we prove that in the overparametrized regime, deep neural network provide universal approximations and can interpolate any data set, as long as the activation function is locally in L1(RR) and not an affine function. Additionally, if the activation function is smooth and such an interpolation networks exists, then the set of parameters which interpolate forms a manifold. Furthermore, we give a characterization of the Hessian of the loss function evaluated at the interpolation points. In the last section, we provide a practical probabilistic method of finding such a point under general conditions on the activation function.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 12 of its 19 references that are in the map. First posted on arXiv 20 Apr 2023.

Builds on

12 of 19 references.

References · 12

  1. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  2. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
  3. 2014Convex Optimization: Algorithms and Complexitycited by 575
  4. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 566
  5. 2015An Introduction to Matrix Concentration Inequalitiescited by 397
  6. 2020Loss landscapes and optimization in over-parameterized non-linear systems and neural networkscited by 233
  7. 2019Universal Approximation with Deep Narrow Networkscited by 200
  8. 2019Towards moderate overparameterization: global convergence guarantees for training shallow neural networkscited by 176
  9. 2021Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolationcited by 132
  10. 2017Approximating Continuous Functions by ReLU Nets of Minimal Widthcited by 122
  11. 2020Minimum Width for Universal Approximationcited by 94
  12. 2018Deep, Skinny Neural Networks are not Universal Approximatorscited by 38

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Just One Layer Norm Guarantees Stable Extrapolationcited by 2
  2. 2026Structural Correspondence and Universal Approximation in Diagonal plus Low-Rank Neural Networkscited by 1
  3. 2026Exact Fixed-Point Constraints in Neural-ODEs with Provable Universalitycited by 1
  4. 2025Robust Deep Network Learning of Nonlinear Regression Tasks by Parametric Leaky Exponential Linear Units (LELUs) and a Diffusion Metriccited by 0