Entropic Confinement and Mode Connectivity in Overparameterized Neural Networks
Not on the map. Why?
Abstract (arXiv)
Modern neural networks exhibit a striking property: basins of attraction in the loss landscape are often connected by low-loss paths, yet optimization dynamics generally remain confined to a single convex basin and rarely explore intermediate points. We resolve this paradox by identifying entropic barriers arising from the interplay between curvature variations along these paths and noise in optimization dynamics. Empirically, we find that curvature systematically rises away from minima, producing effective forces that bias noisy dynamics back toward the endpoints - even when the loss remains nearly flat. These barriers persist longer than energetic barriers, shaping the late-time localization of solutions in parameter space. Our results highlight the role of curvature-induced entropic forces in governing both connectivity and confinement in deep learning landscapes.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 30 references that are in the map. First posted on arXiv 6 Dec 2025.
Builds on
24 of 30 references.
References · 24
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2016Wide Residual Networkscited by 3,070
- 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
- 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
- 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
- 2021Robust fine-tuning of zero-shot modelscited by 797
- 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 637
- 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 588
- 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 471
- 2017Sharp Minima Can Generalize For Deep Netscited by 466
- 2021Gradient Descent on Neural Networks Typically Occurs at the Edge of Stabilitycited by 427
- 2018Essentially No Barriers in Neural Network Energy Landscapecited by 330
12 more
- 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 300
- 2019PyHessian: Neural Networks Through the Lens of the Hessiancited by 292
- 2019Model Fusion via Optimal Transportcited by 219
- 2017A Bayesian Perspective on Generalization and Stochastic Gradient Descentcited by 135
- 2020On the Generalization Benefit of Noise in Stochastic Gradient Descentcited by 82
- 2022Linear Connectivity Reveals Generalization Strategiescited by 56
- 2023Going Beyond Linear Mode Connectivity: The Layerwise Linear Feature Connectivitycited by 50
- 2018Comparing Dynamics: Deep Neural Networks versus Glassy Systemscited by 42
- 2020Noise and Fluctuation of Finite Learning Rate Stochastic Gradient Descentcited by 37
- 2019How noise affects the Hessian spectrum in overparameterized neural networkscited by 17
- 2025The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditionscited by 9
- 2024Landscaping Linear Mode Connectivitycited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.