mlsanityData to 7 Oct 2026

Convergence of Deep Convolutional Neural Networks

arXiv 2109.13542 ↗First posted 28 Sep 2021cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Convergence of deep neural networks as the depth of the networks tends to infinity is fundamental in building the mathematical foundation for deep learning. In a previous study, we investigated this question for deep ReLU networks with a fixed width. This does not cover the important convolutional neural networks where the widths are increasing from layer to layer. For this reason, we first study convergence of general ReLU networks with increasing widths and then apply the results obtained to deep convolutional neural networks. It turns out the convergence reduces to convergence of infinite products of matrices with increasing sizes, which has not been considered in the literature. We establish sufficient conditions for convergence of such infinite products of matrices. Based on the conditions, we present sufficient conditions for piecewise convergence of general deep ReLU networks with increasing widths, and as well as pointwise convergence of deep ReLU convolutional neural networks.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 15 of its 27 references that are in the map. First posted on arXiv 28 Sep 2021.

Builds on

15 of 27 references.

References · 15

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Error bounds for approximations with deep ReLU networkscited by 567
  3. 2018Universality of Deep Convolutional Neural Networkscited by 176
  4. 2016Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Reviewcited by 155
  5. 2018Universal approximations of invariant maps by neural networkscited by 127
  6. 2019Deep Network Approximation Characterized by Number of Neuronscited by 120
  7. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 95
  8. 2019Lipschitz Certificates for Layered Network Structures Driven by Averaged Activation Operatorscited by 54
  9. 2020The gap between theory and practice in function approximation with deep neural networkscited by 53
  10. 2019Error bounds for deep ReLU networks using the Kolmogorov--Arnold superposition theoremcited by 49
  11. 2019Nonlinear Approximation and (Deep) ReLU Networkscited by 45
  12. 2018Exponential Convergence of the Deep Neural Network Approximation for Analytic Functionscited by 38
3 more
  1. 2018Equivalence of approximation by convolutional neural networks and fully-connected networkscited by 38
  2. 2021Convergence of Deep ReLU Networkscited by 14
  3. 2020A Mathematical Introduction to Generative Adversarial Nets (GAN)cited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 5; 2024: 4; 2025: 0; 2026: 1.

13 papers.

Sort:

  1. 2023Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunitiescited by 21
  2. 2024Addressing Spectral Bias of Deep Neural Networks by Multi-Grade Deep Learningcited by 14
  3. 2023Multi-Grade Deep Learningcited by 12
  4. 2022Sparse Deep Neural Network for Nonlinear Partial Differential Equationscited by 5
  5. 2023Uniform Convergence of Deep Neural Networks with Lipschitz Continuous Activation Functions and Variable Widthscited by 5
  6. 2023Multi-Grade Deep Learning for Partial Differential Equations with Applications to the Burgers Equationcited by 5
  7. 2024Deep Neural Network Solutions for Oscillatory Fredholm Integral Equationscited by 5
  8. 2023Successive Affine Learning for Deep Neural Networkscited by 4
  9. 2024Hypothesis Spaces for Deep Learningcited by 4
  10. 2024Advancing the Understanding of Fixed Point Iterations in Deep Neural Networks: A Detailed Analytical Studycited by 4
  11. 2022Convergence Analysis of Deep Residual Networkscited by 1
  12. 2022Convergence of Deep Neural Networks with General Activation Functions and Poolingcited by 0
1 more
  1. 2026Neural Approximation by Function Composition: Rigidity and Doubly Exponential Convergencecited by 0