# Identifying Generalization Properties in Neural Networks

arXiv: https://arxiv.org/abs/1809.07402 · HTML: https://mlsanity.com/paper/1809.07402/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2018.

## Built on by (most cited first, 15 of 15)

- [Deep Model Fusion: A Survey](https://mlsanity.com/paper/2309.15698/) (2023)
- [Anticorrelated Noise Injection for Improved Generalization](https://mlsanity.com/paper/2202.02831/) (2022)
- [ALLSH: Active Learning Guided by Local Sensitivity and Hardness](https://mlsanity.com/paper/2205.04980/) (2022)
- [Effective dimension of machine learning models](https://mlsanity.com/paper/2112.04807/) (2021)
- [Self-Destructive Language Model](https://mlsanity.com/paper/2505.12186/) (2025)
- [Optimization Variance: Exploring Generalization Properties of DNNs](https://mlsanity.com/paper/2106.01714/) (2021)
- [Minimum sharpness: Scale-invariant parameter-robustness of neural networks](https://mlsanity.com/paper/2106.12612/) (2021)
- [BN-invariant sharpness regularizes the training model to better generalization](https://mlsanity.com/paper/2101.02944/) (2021)
- [Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules](https://mlsanity.com/paper/2206.00823/) (2022)
- [A Method for Enhancing Generalization of Adam by Multiple Integrations](https://mlsanity.com/paper/2412.12473/) (2024)
- [Attack-Agnostic Adversarial Detection](https://mlsanity.com/paper/2206.00489/) (2022)
- [Dimension-free uniform concentration bound for logistic regression](https://mlsanity.com/paper/2405.18055/) (2024)
- [LRTuner: A Learning Rate Tuner for Deep Neural Networks](https://mlsanity.com/paper/2105.14526/) (2021)
- [DeepKnowledge: Generalisation-Driven Deep Learning Testing](https://mlsanity.com/paper/2403.16768/) (2024)
- [IKUN: Initialization to Keep snn training and generalization great with sUrrogate-stable variaNce](https://mlsanity.com/paper/2411.18250/) (2024)

## Abstract (arXiv)

While it has not yet been proven, empirical evidence suggests that model generalization is related to local properties of the optima which can be described via the Hessian. We connect model generalization with the local property of a solution under the PAC-Bayes paradigm. In particular, we prove that model generalization ability is related to the Hessian, the higher-order "smoothness" terms characterized by the Lipschitz constant of the Hessian, and the scales of the parameters. Guided by the proof, we propose a metric to score the generalization capability of the model, as well as an algorithm that optimizes the perturbed model accordingly.