mlsanityData to 14 Oct 2026

Sharp Minima Can Generalize: A Loss Landscape Perspective On Data

arXiv 2511.04808 ↗First posted 6 Nov 2025cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

The volume hypothesis suggests deep learning is effective because it is likely to find flat minima due to their large volumes, and flat minima generalize well. This picture does not explain the role of large datasets in generalization. Measuring minima volumes under varying amounts of training data reveals sharp minima which generalize well exist, but are unlikely to be found due to their small volumes. Increasing data changes the loss landscape, such that previously small generalizing minima become (relatively) large.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 18 of its 31 references that are in the map. First posted on arXiv 6 Nov 2025.

Builds on

18 of 31 references.

References · 18

  1. 2020Scaling Laws for Neural Language Modelscited by 6,586
  2. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  3. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  4. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  5. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  6. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  7. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 629
  8. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 588
  9. 2019Fantastic Generalization Measures and Where to Find Themcited by 513
  10. 2017Sharp Minima Can Generalize For Deep Netscited by 466
  11. 2019Implicit Regularization in Deep Matrix Factorizationcited by 405
  12. 2017Train longer, generalize better: closing the generalization gap in large batch training of neural networkscited by 279
6 more
  1. 2023A Modern Look at the Relationship between Sharpness and Generalizationcited by 92
  2. 2021Relating Adversarially Robust Generalization to Flat Minimacited by 57
  3. 2025Deep Learning is Not So Mysterious or Differentcited by 56
  4. 2023Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalizationcited by 39
  5. 2024The Complexity Dynamics of Grokkingcited by 22
  6. 2025Estimating the Probability of Sampling a Trained Neural Network at Randomcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026HeterCSI: Channel-Adaptive Heterogeneous CSI Pretraining Framework for Generalized Wireless Foundation Modelscited by 4
  2. 2026Revisiting the Volume Hypothesiscited by 1
  3. 2026Neural Network Quantization by Learning Low-Loss Subspacescited by 0