mlsanityData to 14 Oct 2026

Global optimality conditions for deep neural networks

arXiv 1707.02444 ↗First posted 8 Jul 2017cs.LG math.OC stat.MLcited by 31

Not on the map. Why?

Abstract (arXiv)

We study the error landscape of deep linear and nonlinear neural networks with the squared error loss. Minimizing the loss of a deep linear neural network is a nonconvex problem, and despite recent progress, our understanding of this loss surface is still incomplete. For deep linear networks, we present necessary and sufficient conditions for a critical point of the risk function to be a global minimum. Surprisingly, our conditions provide an efficiently checkable test for global optimality, while such tests are typically intractable in nonconvex optimization. We further extend these results to deep nonlinear neural networks and prove similar sufficient conditions for global optimality, albeit in a more limited function space setting.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Jul 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 12; 2022: 6; 2023: 3; 2024: 2; 2025: 6; 2026: 2.

31 papers.

Sort:

  1. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 227
  2. 2022Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscapecited by 39
  3. 2021Overparameterization of deep ResNet: zero loss and mean-field analysiscited by 24
  4. 2021The loss landscape of deep linear neural networks: a second-order analysiscited by 24
  5. 2021Convergence of gradient descent for learning linear neural networkscited by 22
  6. 2021Noether: The More Things Change, the More Stay the Samecited by 19
  7. 2025Saddle-to-Saddle Dynamics Explains A Simplicity Bias Across Neural Network Architecturescited by 15
  8. 2021On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regimecited by 12
  9. 2021When Are Solutions Connected in Deep Networks?cited by 11
  10. 2024Only Strict Saddles in the Energy Landscape of Predictive Coding Networks?cited by 10
  11. 2023Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosioncited by 8
  12. 2021Spurious Local Minima Are Common for Deep Neural Networks with Piecewise Linear Activationscited by 7
19 more
  1. 2022Piecewise Linear Neural Networks and Deep Learningcited by 7
  2. 2023Critical Points and Convergence Analysis of Generative Deep Linear Networks Trained with Bures-Wasserstein Losscited by 7
  3. 2021A Convergence Theory Towards Practical Over-parameterized Deep Neural Networkscited by 4
  4. 2024Exploring Loss Landscapes through the Lens of Spin Glass Theorycited by 4
  5. 2025Distributionally Robust Optimization via Diffusion Ambiguity Modelingcited by 4
  6. 2025Backward Oversmoothing: why is it hard to train deep Graph Neural Networks?cited by 3
  7. 2021The Landscape of Multi-Layer Linear Neural Network From the Perspective of Algebraic Geometrycited by 2
  8. 2021Analysis and Optimisation of Bellman Residual Errors with Neural Function Approximationcited by 2
  9. 2022Memorization-Dilation: Modeling Neural Collapse Under Label Noisecited by 2
  10. 2023Transferring Learning Trajectories of Neural Networkscited by 2
  11. 2025Non-Adaptive Adversarial Face Generationcited by 2
  12. 2022Parameter Convex Neural Networkscited by 1
  13. 2022A Dynamics Theory of Implicit Regularization in Deep Low-Rank Matrix Factorizationcited by 1
  14. 2025Learnable Game-theoretic Policy Optimization for Data-centric Self-explanation Rationalizationcited by 1
  15. 2026Structural Correspondence and Universal Approximation in Diagonal plus Low-Rank Neural Networkscited by 1
  16. 2021Training Deep Neural Networks via Branch-and-Boundcited by 0
  17. 2022Complexity from Adaptive-Symmetries Breaking: Global Minima in the Statistical Mechanics of Deep Neural Networkscited by 0
  18. 2025Computing frustration and near-monotonicity in deep neural networkscited by 0
  19. 2026Benign Loss Landscapes Can Coexist with Worst-Case Hardnesscited by 0