mlsanityData to 7 Oct 2026

On the loss landscape of a class of deep neural networks with no bad local valleys

arXiv 1809.10749 ↗First posted 27 Sep 2018cs.LG cs.AI cs.CVcited by 38

Not on the map. Why?

Abstract (arXiv)

We identify a class of over-parameterized deep neural networks with standard activation functions and cross-entropy loss which provably have no bad local valley, in the sense that from any point in parameter space there exists a continuous path on which the cross-entropy loss is non-increasing and gets arbitrarily close to zero. This implies that these networks have no sub-optimal strict local minima.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Sep 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 11; 2023: 5; 2024: 6; 2025: 3; 2026: 3.

38 papers.

Sort:

  1. 2021The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networkscited by 285
  2. 2022REPAIR: REnormalizing Permuted Activations for Interpolation Repaircited by 119
  3. 2021What Happens after SGD Reaches Zero Loss? --A Mathematical Frameworkcited by 106
  4. 2023Deep Model Fusion: A Surveycited by 89
  5. 2022Mechanistic Mode Connectivitycited by 50
  6. 2023Going Beyond Linear Mode Connectivity: The Layerwise Linear Feature Connectivitycited by 50
  7. 2023Catapults in SGD: spikes in the training loss and their impact on generalization through feature learningcited by 29
  8. 2024On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigmcited by 25
  9. 2022Boosting vision transformers for image retrievalcited by 19
  10. 2024Merging Text Transformer Models from Different Initializationscited by 16
  11. 2022Wasserstein Barycenter-based Model Fusion and Linear Mode Connectivity of Neural Networkscited by 14
  12. 2022On the generalization of learning algorithms that do not convergecited by 9
26 more
  1. 2024Exploring Neural Network Landscapes: Star-Shaped and Geodesic Connectivitycited by 9
  2. 2021A Note on Connectivity of Sublevel Sets in Deep Learningcited by 8
  3. 2022On the Omnipresence of Spurious Local Minima in Certain Neural Network Training Problemscited by 8
  4. 2022When Expressivity Meets Trainability: Fewer than n Neurons Can Workcited by 8
  5. 2024Analysis of Linear Mode Connectivity via Permutation-Based Weight Matching: With Insights into Other Permutation Search Methodscited by 8
  6. 2021Spurious Local Minima Are Common for Deep Neural Networks with Piecewise Linear Activationscited by 7
  7. 2021A Gradient Estimator for Time-Varying Electrical Networks with Non-Linear Dissipationcited by 6
  8. 2021What training reveals about neural network complexitycited by 5
  9. 2021Achieving Small Test Error in Mildly Overparameterized Neural Networkscited by 3
  10. 2021Theoretical Exploration of Flexible Transmitter Modelcited by 3
  11. 2025Analyzing the Role of Permutation Invariance in Linear Mode Connectivitycited by 2
  12. 2025Do We Really Need Permutations? Impact of Model Width on Linear Mode Connectivitycited by 2
  13. 2021WGAN with an Infinitely Wide Generator Has No Spurious Stationary Pointscited by 1
  14. 2022Deep learning, stochastic gradient descent and diffusion mapscited by 1
  15. 2024Visualizing, Rethinking, and Mining the Loss Landscape of Deep Neural Networkscited by 1
  16. 2024Geometrical structures of digital fluctuations in parameter space of neural networks trained with adaptive momentum optimizationcited by 1
  17. 2025SE-Merging: A Self-Enhanced Approach for Dynamic Model Mergingcited by 1
  18. 2026Visualizing the loss landscapes of physics-informed neural networkscited by 1
  19. 2021Neural Networks with Complex-Valued Weights Have No Spurious Local Minimacited by 0
  20. 2022Global Convergence Analysis of Deep Linear Networks with A One-neuron Layercited by 0
  21. 2022Complexity from Adaptive-Symmetries Breaking: Global Minima in the Statistical Mechanics of Deep Neural Networkscited by 0
  22. 2022PRANC: Pseudo RAndom Networks for Compacting deep modelscited by 0
  23. 2023Subspace-Configurable Networkscited by 0
  24. 2023Neuro-Visualizer: An Auto-encoder-based Loss Landscape Visualization Methodcited by 0
  25. 2026Beyond Structural Symmetries: Linear Mode Connectivity via Neuron Identifiabilitycited by 0
  26. 2026Benign Loss Landscapes Can Coexist with Worst-Case Hardnesscited by 0