# Negative eigenvalues of the Hessian in deep neural networks

arXiv: https://arxiv.org/abs/1902.02366 · HTML: https://mlsanity.com/paper/1902.02366/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Feb 2019.

## Built on by (most cited first, 20 of 20)

- [Escaping Saddle Points for Effective Generalization on Class-Imbalanced Data](https://mlsanity.com/paper/2212.13827/) (2022)
- [Symmetry Induces Structure and Constraint of Learning](https://mlsanity.com/paper/2309.16932/) (2023)
- [Revisiting the Fragility of Influence Functions](https://mlsanity.com/paper/2303.12922/) (2023)
- [Visualizing high-dimensional loss landscapes with Hessian directions](https://mlsanity.com/paper/2208.13219/) (2022)
- [Towards Quantifying the Hessian Structure of Neural Networks](https://mlsanity.com/paper/2505.02809/) (2025)
- [Parameter Symmetry Potentially Unifies Deep Learning Theory](https://mlsanity.com/paper/2502.05300/) (2025)
- [SGD with a Constant Large Learning Rate Can Converge to Local Maxima](https://mlsanity.com/paper/2107.11774/) (2021)
- [Nesterov acceleration in benignly non-convex landscapes](https://mlsanity.com/paper/2410.08395/) (2024)
- [Learnable Activation Functions in Physics-Informed Neural Networks for Solving Partial Differential Equations](https://mlsanity.com/paper/2411.15111/) (2024)
- [Diffusion Curvature for Estimating Local Curvature in High Dimensional Data](https://mlsanity.com/paper/2206.03977/) (2022)
- [Improving Robust Generalization by Direct PAC-Bayesian Bound Minimization](https://mlsanity.com/paper/2211.12624/) (2022)
- [Unveiling the Hessian's Connection to the Decision Boundary](https://mlsanity.com/paper/2306.07104/) (2023)
- [Curvature-informed multi-task learning for graph networks](https://mlsanity.com/paper/2208.01684/) (2022)
- [A qualitative difference between gradient flows of convex functions in finite- and infinite-dimensional Hilbert spaces](https://mlsanity.com/paper/2310.17610/) (2023)
- [Can Hessian-Based Insights Support Fault Diagnosis in Attention-based Models?](https://mlsanity.com/paper/2506.07871/) (2025)
- [Visualizing, Rethinking, and Mining the Loss Landscape of Deep Neural Networks](https://mlsanity.com/paper/2405.12493/) (2024)
- [Visualizing the loss landscapes of physics-informed neural networks](https://mlsanity.com/paper/2602.05849/) (2026)
- [Characterizing Optimizer-Dependent Training Dynamics Through Hessian Eigenvector Displacement and Localization](https://mlsanity.com/paper/2606.30226/) (2026)
- [Complexity from Adaptive-Symmetries Breaking: Global Minima in the Statistical Mechanics of Deep Neural Networks](https://mlsanity.com/paper/2201.07934/) (2022)
- [On Training Implicit Meta-Learning With Applications to Inductive Weighing in Consistency Regularization](https://mlsanity.com/paper/2310.18741/) (2023)

## Abstract (arXiv)

The loss function of deep networks is known to be non-convex but the precise nature of this nonconvexity is still an active area of research. In this work, we study the loss landscape of deep networks through the eigendecompositions of their Hessian matrix. In particular, we examine how important the negative eigenvalues are and the benefits one can observe in handling them appropriately.