# On the Benefit of Width for Neural Networks: Disappearance of Bad Basins

arXiv: https://arxiv.org/abs/1812.11039 · HTML: https://mlsanity.com/paper/1812.11039/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Dec 2018.

## Built on by (most cited first, 16 of 16)

- [NTK-SAP: Improving neural network pruning by aligning training dynamics](https://mlsanity.com/paper/2304.02840/) (2023)
- [On the Omnipresence of Spurious Local Minima in Certain Neural Network Training Problems](https://mlsanity.com/paper/2202.12262/) (2022)
- [When Expressivity Meets Trainability: Fewer than n Neurons Can Work](https://mlsanity.com/paper/2210.12001/) (2022)
- [Spurious Local Minima Are Common for Deep Neural Networks with Piecewise Linear Activations](https://mlsanity.com/paper/2102.13233/) (2021)
- [Generalization Performance of Empirical Risk Minimization on Over-parameterized Deep ReLU Nets](https://mlsanity.com/paper/2111.14039/) (2021)
- [Global L2 minimization at uniform exponential rate via geometrically adapted gradient descent in Deep Learning](https://mlsanity.com/paper/2311.15487/) (2023)
- [Geometry of Critical Sets and Existence of Saddle Branches for Two-layer Neural Networks](https://mlsanity.com/paper/2405.17501/) (2024)
- [Analyzing the Role of Permutation Invariance in Linear Mode Connectivity](https://mlsanity.com/paper/2503.06001/) (2025)
- [Do We Really Need Permutations? Impact of Model Width on Linear Mode Connectivity](https://mlsanity.com/paper/2510.08023/) (2025)
- [WGAN with an Infinitely Wide Generator Has No Spurious Stationary Points](https://mlsanity.com/paper/2102.07541/) (2021)
- [Component-based Sketching for Deep ReLU Nets](https://mlsanity.com/paper/2409.14174/) (2024)
- [Global Convergence Analysis of Deep Linear Networks with A One-neuron Layer](https://mlsanity.com/paper/2201.02761/) (2022)
- [A Local Convergence Theory for the Stochastic Gradient Descent Method in Non-Convex Optimization With Non-isolated Local Minima](https://mlsanity.com/paper/2203.10973/) (2022)
- [Architecture independent generalization bounds for overparametrized deep ReLU networks](https://mlsanity.com/paper/2504.05695/) (2025)
- [Uncovering Critical Sets of Deep Neural Networks via Sample-Independent Critical Lifting](https://mlsanity.com/paper/2505.13582/) (2025)
- [A Geometric Characterization of the Stationary Plateau for Two-Layer Neural Networks](https://mlsanity.com/paper/2606.04327/) (2026)

## Abstract (arXiv)

Wide networks are often believed to have a nice optimization landscape, but what rigorous results can we prove? To understand the benefit of width, it is important to identify the difference between wide and narrow networks. In this work, we prove that from narrow to wide networks, there is a phase transition from having sub-optimal basins to no sub-optimal basins. Specifically, we prove two results: on the positive side, for any continuous activation functions, the loss surface of a class of wide networks has no sub-optimal basins, where "basin" is defined as the set-wise strict local minimum; on the negative side, for a large class of networks with width below a threshold, we construct strict local minima that are not global. These two results together show the phase transition from narrow to wide networks.