mlsanityData to 14 Oct 2026

Universality of Deep Convolutional Neural Networks

arXiv 1805.10769 ↗First posted 28 May 2018cs.LG stat.MLcited by 176

Not on the map. Why?

Abstract (arXiv)

Deep learning has been widely applied and brought breakthroughs in speech recognition, computer vision, and many other domains. The involved deep neural network architectures and computational issues have been well studied in machine learning. But there lacks a theoretical foundation for understanding the approximation or generalization ability of deep learning methods generated by the network architectures such as deep convolutional neural networks having convolutional structures. Here we show that a deep convolutional neural network (CNN) is universal, meaning that it can be used to approximate any continuous function to an arbitrary accuracy when the depth of the neural network is large enough. This answers an open question in learning theory. Our quantitative estimate, given tightly in terms of the number of free parameters to be computed, verifies the efficiency of deep CNNs in dealing with large dimensional data. Our study also demonstrates the role of convolutions in deep CNNs.

Key facts. Cited by 176 papers in the map (data through 14 Oct 2026). First posted on arXiv 28 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 32; 2022: 37; 2023: 37; 2024: 33; 2025: 22; 2026: 15.

176 papers.

Sort:

By area:

  1. 2022Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practicecited by 136
  2. 2021Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformerscited by 102
  3. 2021Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advancescited by 54
  4. 2024Transformers are Universal In-context Learnerscited by 43
  5. 2023When Does Optimizing a Proper Loss Yield Calibration?cited by 36
  6. 2022Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problemscited by 34
  7. 2023Deep Network Approximation: Beyond ReLU to Diverse Activation Functionscited by 33
  8. 2021Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networkscited by 32
  9. 2021Universal Approximation Under Constraints is Possible with Transformerscited by 32
  10. 2022Likelihood-Free Parameter Estimation with Neural Bayes Estimatorscited by 32
  11. 2021Theory of Deep Convolutional Neural Networks III: Approximating Radial Functionscited by 22
  12. 2021The Impact of Machine Learning on 2D/3D Registration for Image-guided Interventions: A Systematic Review and Perspectivecited by 22
38 more
  1. 2022Stability and Generalization Analysis of Gradient Methods for Shallow Neural Networkscited by 21
  2. 2023Theoretical Analysis of Inductive Biases in Deep Convolutional Networkscited by 21
  3. 2024Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Feature Modelcited by 19
  4. 2021Learning optimal multigrid smoothers via neural networkscited by 17
  5. 2022Neural Network Architecture Beyond Width and Depthcited by 17
  6. 2023PottsMGNet: A Mathematical Explanation of Encoder-Decoder Based Neural Networkscited by 17
  7. 2021Universal Consistency of Deep Convolutional Neural Networkscited by 16
  8. 2021Convergence of Deep ReLU Networkscited by 15
  9. 2021Approximation Properties of Deep ReLU CNNscited by 15
  10. 2023A Brief Survey on the Approximation Theory for Sequence Modellingcited by 15
  11. 2023Optimal rates of approximation by shallow ReLUk neural networks and applications to nonparametric regressioncited by 15
  12. 2023Rethinking Boundary Discontinuity Problem for Oriented Object Detectioncited by 15
  13. 2024A Survey on Universal Approximation Theoremscited by 15
  14. 2021Convolutional Neural Network Compression through Generalized Kronecker Product Decompositioncited by 14
  15. 2024Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Losscited by 14
  16. 2024Addressing Spectral Bias of Deep Neural Networks by Multi-Grade Deep Learningcited by 14
  17. 2021Exploiting epistemic uncertainty of the deep learning models to generate adversarial samplescited by 13
  18. 2021Convergence of Deep Convolutional Neural Networkscited by 13
  19. 2023Neural Bayes Estimators for Irregular Spatial Data using Graph Neural Networkscited by 13
  20. 2021Deep Network Approximation in Terms of Intrinsic Parameterscited by 12
  21. 2022Small Transformers Compute Universal Metric Embeddingscited by 12
  22. 2022Minimal Width for Universal Property of Deep RNNcited by 12
  23. 2023Rates of Approximation by ReLU Shallow Neural Networkscited by 12
  24. 2022Universality of Group Convolutional Neural Networks Based on Ridgelet Analysis on Groupscited by 11
  25. 2023Data pruning and neural scaling laws: fundamental limitations of score-based algorithmscited by 11
  26. 2021Scaling Up Bayesian Uncertainty Quantification for Inverse Problems using Deep Neural Networkscited by 10
  27. 2021Approximation Theory of Convolutional Architectures for Time Series Modellingcited by 10
  28. 2024CDMAD: Class-Distribution-Mismatch-Aware Debiasing for Class-Imbalanced Semi-Supervised Learningcited by 10
  29. 2024On the rates of convergence for learning with convolutional neural networkscited by 10
  30. 2021AngularGrad: A New Optimization Technique for Angular Convergence of Convolutional Neural Networkscited by 9
  31. 2022Deep Neural Network Approximation of Invariant Functions through Dynamical Systemscited by 9
  32. 2022Non-Convergence and Limit Cycles in the Adam optimizercited by 9
  33. 2022Learning Ability of Interpolating Deep Convolutional Neural Networkscited by 9
  34. 2023Expressivity and Approximation Properties of Deep Neural Networks with ReLUk Activationcited by 9
  35. 2024A Characterization Theorem for Equivariant Networks with Point-wise Activationscited by 9
  36. 2023Approximation of Nonlinear Functionals Using Deep ReLU Networkscited by 8
  37. 2023Understanding the Initial Condensation of Convolutional Neural Networkscited by 8
  38. 2025Block-Biased Mamba for Long-Range Sequence Processingcited by 8

50 most cited of 176 All 176 (JSON)