mlsanityData to 7 Oct 2026

Understanding Generalization through Visualizations

arXiv 1906.03291 ↗First posted 7 Jun 2019cs.LG cs.NE stat.MLcited by 54

Not on the map. Why?

Abstract (arXiv)

The power of neural networks lies in their ability to generalize to unseen data, yet the underlying reasons for this phenomenon remain elusive. Numerous rigorous attempts have been made to explain generalization, but available bounds are still quite loose, and analysis does not always lead to true understanding. The goal of this work is to make generalization more intuitive. Using visualization methods, we discuss the mystery of generalization, the geometry of loss landscapes, and how the curse (or, rather, the blessing) of dimensionality causes optimizers to settle into minima that generalize well.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 11; 2023: 11; 2024: 10; 2025: 6; 2026: 5.

54 papers.

Sort:

  1. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 388
  2. 2021The Intrinsic Dimension of Images and Its Impact on Learningcited by 307
  3. 2021Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensemblingcited by 75
  4. 2021Stochastic Training is Not Necessary for Generalizationcited by 63
  5. 2023The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learningcited by 63
  6. 2022Linear Connectivity Reveals Generalization Strategiescited by 55
  7. 2025Deep Learning is Not So Mysterious or Differentcited by 55
  8. 2022On Uncertainty, Tempering, and Data Augmentation in Bayesian Classificationcited by 46
  9. 2022Pre-Train Your Loss: Easy Bayesian Transfer Learning with Informative Priorscited by 43
  10. 2023Rethinking Confidence Calibration for Failure Predictioncited by 43
  11. 2022How Much Data Are Augmentations Worth? An Investigation into Scaling Laws, Invariance, and Implicit Regularizationcited by 40
  12. 2021Preventing Unauthorized Use of Proprietary Data: Poisoning for Secure Dataset Releasecited by 35
38 more
  1. 2024Out-of-Domain Generalization in Dynamical Systems Reconstructioncited by 33
  2. 2022MLPInit: Embarrassingly Simple GNN Training Acceleration with MLP Initializationcited by 31
  3. 2024Neural Networks Learn Statistics of Increasing Complexitycited by 30
  4. 2021Test time Adaptation through Perturbation Robustnesscited by 28
  5. 2021Active Learning at the ImageNet Scalecited by 28
  6. 2021Discovering Distinctive "Semantics" in Super-Resolution Networkscited by 20
  7. 2023DART: Diversify-Aggregate-Repeat Training Improves Generalization of Neural Networkscited by 20
  8. 2023Simplifying Neural Network Training Under Class Imbalancecited by 18
  9. 2023Sharpness-Aware Minimization Revisited: Weighted Sharpness as a Regularization Termcited by 16
  10. 2022Improving the Sample Efficiency of Prompt Tuning with Domain Adaptationcited by 15
  11. 2022K-SAM: Sharpness-Aware Minimization at the Speed of SGDcited by 10
  12. 2021The Uncanny Similarity of Recurrence and Depthcited by 9
  13. 2024Revisiting Confidence Estimation: Towards Reliable Failure Predictioncited by 9
  14. 2021On The State of Data In Computer Vision: Human Annotations Remain Indispensable for Developing Deep Learning Modelscited by 6
  15. 2022A picture of the space of typical learnable taskscited by 6
  16. 2023Entropy-MCMC: Sampling from Flat Basins with Easecited by 6
  17. 2023Perspectives on the State and Future of Deep Learning - 2023cited by 4
  18. 2024Just How Flexible are Neural Networks in Practice?cited by 3
  19. 2025Estimating the Probability of Sampling a Trained Neural Network at Randomcited by 3
  20. 2024Geometry of Critical Sets and Existence of Saddle Branches for Two-layer Neural Networkscited by 2
  21. 2024Bias of Stochastic Gradient Descent or the Architecture: Disentangling the Effects of Overparameterization of Neural Networkscited by 2
  22. 2024Can Optimization Trajectories Explain Multi-Task Transfer?cited by 2
  23. 2025Kolmogorov-Arnold Attention: Is Learnable Attention Better For Vision Transformers?cited by 2
  24. 2023Probing optimisation in physics-informed neural networkscited by 1
  25. 2023Revealing the Underlying Patterns: Investigating Dataset Similarity, Performance, and Generalizationcited by 1
  26. 2024Visualizing, Rethinking, and Mining the Loss Landscape of Deep Neural Networkscited by 1
  27. 2025MOCLIP: A Foundation Model for Large-Scale Nanophotonic Inverse Designcited by 1
  28. 2026Beyond LoRA: Is Sparsity-Induced Adaptation Better?cited by 1
  29. 2026Revisiting the Volume Hypothesiscited by 1
  30. 2026Dataset Complexity Shapes Finite-Distance Loss Geometry in Neural Networkscited by 1
  31. 2021XConv: Low-memory stochastic backpropagation for convolutional layerscited by 0
  32. 2022AutoProtoNet: Interpretability for Prototypical Networkscited by 0
  33. 2022Empirical investigations on WVA structural issuescited by 0
  34. 2022Correlation of the importances of neural network weights calculated by modern methods of overcoming catastrophic forgettingcited by 0
  35. 2023G-TRACER: Expected Sharpness Optimizationcited by 0
  36. 2023Neuro-Visualizer: An Auto-encoder-based Loss Landscape Visualization Methodcited by 0
  37. 2024Insights into the Lottery Ticket Hypothesis and Iterative Magnitude Pruningcited by 0
  38. 2024OP-LoRA: The Blessing of Dimensionalitycited by 0

50 most cited of 54 All 54 (JSON)