On the curvature of the loss landscape
Not on the map. Why?
Abstract (arXiv)
One of the main challenges in modern deep learning is to understand why such over-parameterized models perform so well when trained on finite data. A way to analyze this generalization concept is through the properties of the associated loss landscape. In this work, we consider the loss landscape as an embedded Riemannian manifold and show that the differential geometric properties of the manifold can be used when analyzing the generalization abilities of a deep net. In particular, we focus on the scalar curvature, which can be computed analytically for our manifold, and show connections to several settings that potentially imply generalization.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 14 of its 23 references that are in the map. First posted on arXiv 10 Jul 2023.
Builds on
14 of 23 references.
References · 14
- 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
- 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
- 2017Sharp Minima Can Generalize For Deep Netscited by 462
- 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 357
- 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 298
- 2019An Investigation into Neural Net Optimization via Hessian Eigenvalue Densitycited by 289
- 2017Three Factors Influencing Minima in SGDcited by 270
- 2022Anticorrelated Noise Injection for Improved Generalizationcited by 41
- 2017Geometry of Optimization and Implicit Regularization in Deep Learningcited by 39
- 2023Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusioncited by 39
- 2020Entropic gradient descent algorithms and wide flat minimacited by 25
- 2019How noise affects the Hessian spectrum in overparameterized neural networkscited by 17
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.