mlsanityData to 14 Oct 2026

Adding One Neuron Can Eliminate All Bad Local Minima

arXiv 1805.08671 ↗First posted 22 May 2018stat.ML cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

One of the main difficulties in analyzing neural networks is the non-convexity of the loss function which may have many bad local minima. In this paper, we study the landscape of neural networks for binary classification tasks. Under mild assumptions, we prove that after adding one special neuron with a skip connection to the output, or one special neuron per layer, every local minimum is a global minimum.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). First posted on arXiv 22 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 9; 2022: 3; 2023: 3; 2024: 2; 2025: 2; 2026: 2.

21 papers.

Sort:

  1. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 227
  2. 2021On the Theory of Implicit Deep Learning: Global Convergence with Implicit Layerscited by 35
  3. 2023NTK-SAP: Improving neural network pruning by aligning training dynamicscited by 30
  4. 2021When Are Solutions Connected in Deep Networks?cited by 11
  5. 2023Global Optimality in Bivariate Gradient-based DAG Learningcited by 10
  6. 2024Exploring Neural Network Landscapes: Star-Shaped and Geodesic Connectivitycited by 9
  7. 2022When Expressivity Meets Trainability: Fewer than n Neurons Can Workcited by 8
  8. 2021Spurious Local Minima Are Common for Deep Neural Networks with Piecewise Linear Activationscited by 7
  9. 2021Achieving Small Test Error in Mildly Overparameterized Neural Networkscited by 3
  10. 2021Theoretical Exploration of Flexible Transmitter Modelcited by 3
  11. 2025Flat Channels to Infinity in Neural Loss Landscapescited by 3
  12. 2025Do We Really Need Permutations? Impact of Model Width on Linear Mode Connectivitycited by 2
9 more
  1. 2021A Comprehensive Study on Optimization Strategies for Gradient Descent In Deep Learningcited by 1
  2. 2021WGAN with an Infinitely Wide Generator Has No Spurious Stationary Pointscited by 1
  3. 2021Training Deep Neural Networks via Branch-and-Boundcited by 0
  4. 2022Global Convergence Analysis of Deep Linear Networks with A One-neuron Layercited by 0
  5. 2022Noise Regularizes Over-parameterized Rank One Matrix Recovery, Provablycited by 0
  6. 2023Blurry Video Compression: A Trade-off between Visual Enhancement and Data Compressioncited by 0
  7. 2024Analysis of the rate of convergence of an over-parametrized convolutional neural network image classifier learned by gradient descentcited by 0
  8. 2026Certified Inference and Training for Deep Equilibrium Networks: A Continuation Framework with Polynomial Complexity Guaranteescited by 0
  9. 2026Removing spurious minima for planar features by skip connectionscited by 0