mlsanityData to 7 Oct 2026

Depth with Nonlinearity Creates No Bad Local Minima in ResNets

arXiv 1810.09038 ↗First posted 21 Oct 2018stat.ML cs.AI cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

In this paper, we prove that depth with nonlinearity creates no bad local minima in a type of arbitrarily deep ResNets with arbitrary nonlinear activation functions, in the sense that the values of all local minima are no worse than the global minimum value of corresponding classical machine-learning models, and are guaranteed to further improve via residual representations. As a result, this paper provides an affirmative answer to an open question stated in a paper in the conference on Neural Information Processing Systems 2018. This paper advances the optimization theory of deep learning only for ResNets and not for other network architectures.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 1; 2023: 1; 2024: 2; 2025: 1; 2026: 1.

17 papers.

Sort:

  1. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 95
  2. 2022Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodologycited by 43
  3. 2021Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neuronscited by 41
  4. 2021On the Theory of Implicit Deep Learning: Global Convergence with Implicit Layerscited by 35
  5. 2021Deep Network Approximation in Terms of Intrinsic Parameterscited by 12
  6. 2021When Are Solutions Connected in Deep Networks?cited by 10
  7. 2021A Recipe for Global Convergence Guarantee in Deep Neural Networkscited by 8
  8. 2021Spurious Local Minima Are Common for Deep Neural Networks with Piecewise Linear Activationscited by 7
  9. 2021ResNEsts and DenseNEsts: Block-based DNN Models with Improved Representation Guaranteescited by 6
  10. 2021Learning distinct features helps, provablycited by 4
  11. 2021Theoretical Exploration of Flexible Transmitter Modelcited by 3
  12. 2023Learning Unorthogonalized Matrices for Rotation Estimationcited by 3
5 more
  1. 2024Mean-Field Analysis for Learning Subspace-Sparse Polynomials with Gaussian Inputcited by 1
  2. 2025System Identification and Control Using Lyapunov-Based Deep Neural Networks without Persistent Excitation: A Concurrent Learning Approachcited by 1
  3. 2021Advances in Electron Microscopy with Deep Learningcited by 0
  4. 2024ExpTest: Loss-Curve Hypothesis Testing for Autonomous Learning-Rate Selection in Deep Neural Networkscited by 0
  5. 2026Removing spurious minima for planar features by skip connectionscited by 0