mlsanityData to 7 Oct 2026

A Note on Connectivity of Sublevel Sets in Deep Learning

arXiv 2101.08576 ↗First posted 21 Jan 2021cs.LG stat.MLcited by 8

Not on the map. Why?

Abstract (arXiv)

It is shown that for deep neural networks, a single wide layer of width N+1 (N being the number of training samples) suffices to prove the connectivity of sublevel sets of the training loss function. In the two-layer setting, the same property may not hold even if one has just one neuron less (i.e. width N can lead to disconnected sublevel sets).

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 10 of its 14 references that are in the map. First posted on arXiv 21 Jan 2021.

Builds on

10 of 14 references.

References · 10

  1. 2014The Loss Surfaces of Multilayer Networkscited by 278
  2. 2016Deep Learning without Poor Local Minimacited by 233
  3. 2016Topology and Geometry of Half-Rectified Network Optimizationcited by 102
  4. 2017The loss surface of deep and wide neural networkscited by 74
  5. 2019Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Netscited by 63
  6. 2019On Connected Sublevel Sets in Deep Learningcited by 51
  7. 2018On the loss landscape of a class of deep neural networks with no bad local valleyscited by 38
  8. 2019Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural Networkscited by 35
  9. 2018Small nonlinearities in activation functions create bad local minima in neural networkscited by 28
  10. 2018Spurious Valleys in Two-layer Neural Network Optimization Landscapescited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 2; 2023: 0; 2024: 0; 2025: 3; 2026: 1.

8 papers.

  1. 2021On the Theory of Implicit Deep Learning: Global Convergence with Implicit Layerscited by 35
  2. 2022Neural Tangent Kernel: A Surveycited by 22
  3. 2021When Are Solutions Connected in Deep Networks?cited by 10
  4. 2022Plateau in Monotonic Linear Interpolation -- A "Biased" View of Loss Landscape for Deep Networkscited by 10
  5. 2025Understanding Mode Connectivity via Parameter Space Symmetrycited by 10
  6. 2025Do We Really Need Permutations? Impact of Model Width on Linear Mode Connectivitycited by 2
  7. 2025Favorability of Loss Landscape with Weight Decay Requires Both Large Overparametrization and Initializationcited by 0
  8. 2026Optimizer-Induced Mode Connectivity: From AdamW to Muoncited by 0