# Information-Theoretic Local Minima Characterization and Regularization

arXiv: https://arxiv.org/abs/1911.08192 · HTML: https://mlsanity.com/paper/1911.08192/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2019.

## Built on by (most cited first, 14 of 14)

- [Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization](https://mlsanity.com/paper/2303.03108/) (2023)
- [Flatness-Aware Minimization for Domain Generalization](https://mlsanity.com/paper/2307.11108/) (2023)
- [A Survey on Evaluation of Out-of-Distribution Generalization](https://mlsanity.com/paper/2403.01874/) (2024)
- [Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias](https://mlsanity.com/paper/2210.02720/) (2022)
- [Reward Fine-Tuning Two-Step Diffusion Models via Learning Differentiable Latent-Space Surrogate Reward](https://mlsanity.com/paper/2411.15247/) (2024)
- [The Hessian perspective into the Nature of Convolutional Neural Networks](https://mlsanity.com/paper/2305.09088/) (2023)
- [AlterSGD: Finding Flat Minima for Continual Learning by Alternative Training](https://mlsanity.com/paper/2107.05804/) (2021)
- [Improving Robust Generalization by Direct PAC-Bayesian Bound Minimization](https://mlsanity.com/paper/2211.12624/) (2022)
- [Transferability Bound Theory: Exploring Relationship between Adversarial Transferability and Flatness](https://mlsanity.com/paper/2311.06423/) (2023)
- [Soft ascent-descent as a stable and flexible alternative to flooding](https://mlsanity.com/paper/2310.10006/) (2023)
- [R'enyi Sharpness: A Novel Sharpness that Strongly Correlates with Generalization](https://mlsanity.com/paper/2510.07758/) (2025)
- [Contractive Diffusion Policies: Robust Action Diffusion via Contractive Score-Based Sampling with Differential Equations](https://mlsanity.com/paper/2601.01003/) (2026)
- [Gradient Regularized Natural Gradients](https://mlsanity.com/paper/2601.18420/) (2026)
- [Doubly Stochastic Models: Learning with Unbiased Label Noises and Inference Stability](https://mlsanity.com/paper/2304.00320/) (2023)

## Abstract (arXiv)

Recent advances in deep learning theory have evoked the study of generalizability across different local minima of deep neural networks (DNNs). While current work focused on either discovering properties of good local minima or developing regularization techniques to induce good local minima, no approach exists that can tackle both problems. We achieve these two goals successfully in a unified manner. Specifically, based on the observed Fisher information we propose a metric both strongly indicative of generalizability of local minima and effectively applied as a practical regularizer. We provide theoretical analysis including a generalization bound and empirically demonstrate the success of our approach in both capturing and improving the generalizability of DNNs. Experiments are performed on CIFAR-10, CIFAR-100 and ImageNet for various network architectures.