mlsanityData to 7 Oct 2026

Uniform Convergence of Deep Neural Networks with Lipschitz Continuous Activation Functions and Variable Widths

arXiv 2306.01692 ↗First posted 2 Jun 2023cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

We consider deep neural networks with a Lipschitz continuous activation function and with weight matrices of variable widths. We establish a uniform convergence analysis framework in which sufficient conditions on weight matrices and bias vectors together with the Lipschitz constant are provided to ensure uniform convergence of the deep neural networks to a meaningful function as the number of their layers tends to infinity. In the framework, special results on uniform convergence of deep neural networks with a fixed width, bounded widths and unbounded widths are presented. In particular, as convolutional neural networks are special deep neural networks with weight matrices of increasing widths, we put forward conditions on the mask sequence which lead to uniform convergence of resulting convolutional neural networks. The Lipschitz continuity assumption on the activation functions allows us to include in our theory most of commonly used activation functions in applications.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 31 of its 52 references that are in the map. First posted on arXiv 2 Jun 2023.

Builds on

31 of 52 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  3. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  4. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  5. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  6. 2017Deep Neural Networks as Gaussian Processescited by 617
  7. 2017Self-Normalizing Neural Networkscited by 591
  8. 2016Error bounds for approximations with deep ReLU networkscited by 567
  9. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
  10. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 314
  11. 2016Exponential expressivity in deep neural networks through transient chaoscited by 260
  12. 2018Universality of Deep Convolutional Neural Networkscited by 176
19 more
  1. 2016Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Reviewcited by 155
  2. 2016Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivitycited by 138
  3. 2019Deep Network Approximation Characterized by Number of Neuronscited by 120
  4. 2019Finite Depth and Width Corrections to the Neural Tangent Kernelcited by 114
  5. 2019Dynamics of Deep Neural Networks and Neural Tangent Hierarchycited by 96
  6. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 95
  7. 2020Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networkscited by 87
  8. 2019Lipschitz Certificates for Layered Network Structures Driven by Averaged Activation Operatorscited by 54
  9. 2019Error bounds for deep ReLU networks using the Kolmogorov--Arnold superposition theoremcited by 49
  10. 2018Exponential Convergence of the Deep Neural Network Approximation for Analytic Functionscited by 38
  11. 2021An error analysis of generative adversarial networks for learning distributionscited by 35
  12. 2018On Lipschitz Bounds of General Convolutional Neural Networkscited by 29
  13. 2020On the Neural Tangent Kernel of Deep Networks with Orthogonal Initializationcited by 21
  14. 2021Convergence of Deep ReLU Networkscited by 14
  15. 2021Convergence of Deep Convolutional Neural Networkscited by 13
  16. 2023Multi-Grade Deep Learningcited by 12
  17. 2020A Mathematical Introduction to Generative Adversarial Nets (GAN)cited by 8
  18. 2023Successive Affine Learning for Deep Neural Networkscited by 4
  19. 2022Convergence Analysis of Deep Residual Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 0; 2026: 1.

5 papers.

  1. 2024Addressing Spectral Bias of Deep Neural Networks by Multi-Grade Deep Learningcited by 14
  2. 2023Multi-Grade Deep Learning for Partial Differential Equations with Applications to the Burgers Equationcited by 5
  3. 2024Hypothesis Spaces for Deep Learningcited by 4
  4. 2024Advancing the Understanding of Fixed Point Iterations in Deep Neural Networks: A Detailed Analytical Studycited by 4
  5. 2026Neural Approximation by Function Composition: Rigidity and Doubly Exponential Convergencecited by 0