# Improving Generalization of Deep Neural Networks by Leveraging Margin Distribution

arXiv: https://arxiv.org/abs/1812.10761 · HTML: https://mlsanity.com/paper/1812.10761/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Dec 2018.

## Built on by (most cited first, 5 of 5)

- [Confidence-aware Contrastive Learning for Selective Classification](https://mlsanity.com/paper/2406.04745/) (2024)
- [Semantic-Rearrangement-Based Multi-Level Alignment for Domain Generalized Segmentation](https://mlsanity.com/paper/2404.13701/) (2024)
- [Depth-induced NTK: Bridging Over-parameterized Neural Networks and Deep Neural Kernels](https://mlsanity.com/paper/2511.05585/) (2025)
- [PENEX: AdaBoost-Inspired Neural Network Regularization](https://mlsanity.com/paper/2510.02107/) (2025)
- [Do More Modalities Always Help? A Geometric Perspective on Missing-Modality Robustness](https://mlsanity.com/paper/2610.04792/) (2026)

## Abstract (arXiv)

Recent research has used margin theory to analyze the generalization performance for deep neural networks (DNNs). The existed results are almost based on the spectrally-normalized minimum margin. However, optimizing the minimum margin ignores a mass of information about the entire margin distribution, which is crucial to generalization performance. In this paper, we prove a generalization upper bound dominated by the statistics of the entire margin distribution. Compared with the minimum margin bounds, our bound highlights an important measure for controlling the complexity, which is the ratio of the margin standard deviation to the expected margin. We utilize a convex margin distribution loss function on the deep neural networks to validate our theoretical results by optimizing the margin ratio. Experiments and visualizations confirm the effectiveness of our approach and the correlation between generalization gap and margin ratio.