# Explaining f-Divergence-Based Regularization via Local Curvature and Sharpness-Aware Minimization

arXiv: https://arxiv.org/abs/2609.09367 · HTML: https://mlsanity.com/paper/2609.09367/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 8 Sep 2026.

## Builds on (most cited first, 16 of 16)

- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [Fisher SAM: Information Geometry and Sharpness Aware Minimisation](https://mlsanity.com/paper/2206.04920/) (2022)
- [How Does Sharpness-Aware Minimization Minimize Sharpness?](https://mlsanity.com/paper/2211.05729/) (2022)
- [Distributional Smoothing with Virtual Adversarial Training](https://mlsanity.com/paper/1507.00677/) (2015)
- [When Optimizing f-divergence is Robust with Label Noise](https://mlsanity.com/paper/2011.03687/) (2020)
- [SAM operates far from home: eigenvalue regularization as a dynamical phenomenon](https://mlsanity.com/paper/2302.08692/) (2023)
- [Adversarial Robustness via Fisher-Rao Regularization](https://mlsanity.com/paper/2106.06685/) (2021)
- [CR-SAM: Curvature Regularized Sharpness-Aware Minimization](https://mlsanity.com/paper/2312.13555/) (2023)
- [Learning Fair Classifiers via Min-Max F-divergence Regularization](https://mlsanity.com/paper/2306.16552/) (2023)
- [X-SHIELD: Regularization for eXplainable Artificial Intelligence](https://mlsanity.com/paper/2404.02611/) (2024)
- [Regularization via f-Divergence: An Application to Multi-Oxide Spectroscopic Analysis](https://mlsanity.com/paper/2502.03755/) (2025)

## Abstract (arXiv)

Divergence-based regularization and Sharpness-Aware Minimization (SAM) are two prominent approaches for improving generalization in deep learning, both motivated by robustness to perturbations. However, their relationship has remained largely unexplored. Building on classical second-order expansions of f-divergences, we show that the two methods are locally consistent under parameter-space perturbations: both induce curvature-sensitive penalties, with divergence regularization yielding a Fisher-weighted quadratic form and SAM penalizing sharpness through the dominant Hessian eigenvalue. For negative log-likelihood objectives with exponential-family output distributions, this correspondence becomes especially transparent, since the Fisher and Gauss-Newton matrices coincide. We further show that the same local geometric perspective extends to input-space perturbations, where divergence-based regularization is defined through transformations of the input. In this setting, the regularizer induces a pullback quadratic form on the input space, providing a more general perturbation framework than standard SAM while preserving the same local sensitivity interpretation. To validate the analysis empirically, we use the asymmetric α-skew Jensen-Shannon divergence (JSD) family as a controlled testbed. Its local curvature coefficient scales as α(1-α) and is maximized at the symmetric point α=tfrac12, which recovers the standard JSD. Loss-landscape visualizations in the input-perturbation regime show that stronger induced curvature penalization is associated with flatter local minima. Experiments on four benchmark datasets further demonstrate that both accuracy and negative log-likelihood are consistently best near this regime of maximal curvature penalization.