mlsanityData to 14 Oct 2026

Negative eigenvalues of the Hessian in deep neural networks

arXiv 1902.02366 ↗First posted 6 Feb 2019cs.LG math.OC stat.MLcited by 20

Not on the map. Why?

Abstract (arXiv)

The loss function of deep networks is known to be non-convex but the precise nature of this nonconvexity is still an active area of research. In this work, we study the loss landscape of deep networks through the eigendecompositions of their Hessian matrix. In particular, we examine how important the negative eigenvalues are and the benefits one can observe in handling them appropriately.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). First posted on arXiv 6 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 5; 2024: 3; 2025: 3; 2026: 2.

20 papers.

Sort:

  1. 2022Escaping Saddle Points for Effective Generalization on Class-Imbalanced Datacited by 28
  2. 2023Symmetry Induces Structure and Constraint of Learningcited by 25
  3. 2023Revisiting the Fragility of Influence Functionscited by 20
  4. 2022Visualizing high-dimensional loss landscapes with Hessian directionscited by 16
  5. 2025Towards Quantifying the Hessian Structure of Neural Networkscited by 13
  6. 2025Parameter Symmetry Potentially Unifies Deep Learning Theorycited by 12
  7. 2021SGD with a Constant Large Learning Rate Can Converge to Local Maximacited by 8
  8. 2024Nesterov acceleration in benignly non-convex landscapescited by 7
  9. 2024Learnable Activation Functions in Physics-Informed Neural Networks for Solving Partial Differential Equationscited by 7
  10. 2022Diffusion Curvature for Estimating Local Curvature in High Dimensional Datacited by 6
  11. 2022Improving Robust Generalization by Direct PAC-Bayesian Bound Minimizationcited by 6
  12. 2023Unveiling the Hessian's Connection to the Decision Boundarycited by 5
8 more
  1. 2022Curvature-informed multi-task learning for graph networkscited by 3
  2. 2023A qualitative difference between gradient flows of convex functions in finite- and infinite-dimensional Hilbert spacescited by 3
  3. 2025Can Hessian-Based Insights Support Fault Diagnosis in Attention-based Models?cited by 2
  4. 2024Visualizing, Rethinking, and Mining the Loss Landscape of Deep Neural Networkscited by 1
  5. 2026Visualizing the loss landscapes of physics-informed neural networkscited by 1
  6. 2026Characterizing Optimizer-Dependent Training Dynamics Through Hessian Eigenvector Displacement and Localizationcited by 1
  7. 2022Complexity from Adaptive-Symmetries Breaking: Global Minima in the Statistical Mechanics of Deep Neural Networkscited by 0
  8. 2023On Training Implicit Meta-Learning With Applications to Inductive Weighing in Consistency Regularizationcited by 0