# On Tighter Generalization Bound for Deep Neural Networks: CNNs, ResNets, and Beyond

arXiv: https://arxiv.org/abs/1806.05159 · HTML: https://mlsanity.com/paper/1806.05159/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jun 2018.

## Built on by (most cited first, 24 of 24)

- [Generalization Error Bounds for Deep Neural Networks Trained by SGD](https://mlsanity.com/paper/2206.03299/) (2022)
- [Adversarial robustness of sparse local Lipschitz predictors](https://mlsanity.com/paper/2202.13216/) (2022)
- [The Sample Complexity of One-Hidden-Layer Neural Networks](https://mlsanity.com/paper/2202.06233/) (2022)
- [LocalDrop: A Hybrid Regularization for Deep Neural Networks](https://mlsanity.com/paper/2103.00719/) (2021)
- [Sparsity-aware generalization theory for deep neural networks](https://mlsanity.com/paper/2307.00426/) (2023)
- [Visualizing the Emergence of Intermediate Visual Patterns in DNNs](https://mlsanity.com/paper/2111.03505/) (2021)
- [Generalization Bounds for Few-Shot Transfer Learning with Pretrained Classifiers](https://mlsanity.com/paper/2212.12532/) (2022)
- [Norm-based Generalization Bounds for Compositionally Sparse Neural Networks](https://mlsanity.com/paper/2301.12033/) (2023)
- [Inferential Wasserstein Generative Adversarial Networks](https://mlsanity.com/paper/2109.05652/) (2021)
- [Leaner Transformers: More Heads, Less Depth](https://mlsanity.com/paper/2505.20802/) (2025)
- [Non-asymptotic Excess Risk Bounds for Classification with Deep Convolutional Neural Networks](https://mlsanity.com/paper/2105.00292/) (2021)
- [How much pre-training is enough to discover a good subnetwork?](https://mlsanity.com/paper/2108.00259/) (2021)
- [Trap of Feature Diversity in the Learning of MLPs](https://mlsanity.com/paper/2112.00980/) (2021)
- [Exploring the Complexity of Deep Neural Networks through Functional Equivalence](https://mlsanity.com/paper/2305.11417/) (2023)
- [The Fair Language Model Paradox](https://mlsanity.com/paper/2410.11985/) (2024)
- [CausalEGM: a general causal inference framework by encoding generative modeling](https://mlsanity.com/paper/2212.05925/) (2022)
- [Toward Sustainable Continual Learning: Detection and Knowledge Repurposing of Similar Tasks](https://mlsanity.com/paper/2210.05751/) (2022)
- [On the Generalization Behavior of Deep Residual Networks From a Dynamical System Perspective](https://mlsanity.com/paper/2602.20921/) (2026)
- [Understanding Adversarial Examples Through Deep Neural Network's Response Surface and Uncertainty Regions](https://mlsanity.com/paper/2107.00003/) (2021)
- [On the Robustness and Generalization of Deep Learning Driven Full Waveform Inversion](https://mlsanity.com/paper/2111.14220/) (2021)
- [Doubly Stochastic Models: Learning with Unbiased Label Noises and Inference Stability](https://mlsanity.com/paper/2304.00320/) (2023)
- [Generalizability of Neural Networks Minimizing Empirical Risk Based on Expressive Ability](https://mlsanity.com/paper/2503.04111/) (2025)
- [Overfitting has a limitation: a model-independent generalization gap bound based on R'enyi entropy](https://mlsanity.com/paper/2506.00182/) (2025)
- [Position: Many generalization measures for deep learning are fragile](https://mlsanity.com/paper/2510.18934/) (2025)

## Abstract (arXiv)

We establish a margin based data dependent generalization error bound for a general family of deep neural networks in terms of the depth and width, as well as the Jacobian of the networks. Through introducing a new characterization of the Lipschitz properties of neural network family, we achieve significantly tighter generalization bounds than existing results. Moreover, we show that the generalization bound can be further improved for bounded losses. Aside from the general feedforward deep neural networks, our results can be applied to derive new bounds for popular architectures, including convolutional neural networks (CNNs) and residual networks (ResNets). When achieving same generalization errors with previous arts, our bounds allow for the choice of larger parameter spaces of weight matrices, inducing potentially stronger expressive ability for neural networks. Numerical evaluation is also provided to support our theory.