# Global optimality conditions for deep neural networks

arXiv: https://arxiv.org/abs/1707.02444 · HTML: https://mlsanity.com/paper/1707.02444/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jul 2017.

## Built on by (most cited first, 30 of 31)

- [A Geometric Analysis of Neural Collapse with Unconstrained Features](https://mlsanity.com/paper/2105.02375/) (2021)
- [Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscape](https://mlsanity.com/paper/2201.08025/) (2022)
- [Overparameterization of deep ResNet: zero loss and mean-field analysis](https://mlsanity.com/paper/2105.14417/) (2021)
- [The loss landscape of deep linear neural networks: a second-order analysis](https://mlsanity.com/paper/2107.13289/) (2021)
- [Convergence of gradient descent for learning linear neural networks](https://mlsanity.com/paper/2108.02040/) (2021)
- [Noether: The More Things Change, the More Stay the Same](https://mlsanity.com/paper/2104.05508/) (2021)
- [Saddle-to-Saddle Dynamics Explains A Simplicity Bias Across Neural Network Architectures](https://mlsanity.com/paper/2512.20607/) (2025)
- [On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regime](https://mlsanity.com/paper/2110.02926/) (2021)
- [When Are Solutions Connected in Deep Networks?](https://mlsanity.com/paper/2102.09671/) (2021)
- [Only Strict Saddles in the Energy Landscape of Predictive Coding Networks?](https://mlsanity.com/paper/2408.11979/) (2024)
- [Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosion](https://mlsanity.com/paper/2310.02012/) (2023)
- [Spurious Local Minima Are Common for Deep Neural Networks with Piecewise Linear Activations](https://mlsanity.com/paper/2102.13233/) (2021)
- [Piecewise Linear Neural Networks and Deep Learning](https://mlsanity.com/paper/2206.09149/) (2022)
- [Critical Points and Convergence Analysis of Generative Deep Linear Networks Trained with Bures-Wasserstein Loss](https://mlsanity.com/paper/2303.03027/) (2023)
- [A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks](https://mlsanity.com/paper/2101.04243/) (2021)
- [Exploring Loss Landscapes through the Lens of Spin Glass Theory](https://mlsanity.com/paper/2407.20724/) (2024)
- [Distributionally Robust Optimization via Diffusion Ambiguity Modeling](https://mlsanity.com/paper/2510.22757/) (2025)
- [Backward Oversmoothing: why is it hard to train deep Graph Neural Networks?](https://mlsanity.com/paper/2505.16736/) (2025)
- [The Landscape of Multi-Layer Linear Neural Network From the Perspective of Algebraic Geometry](https://mlsanity.com/paper/2102.04338/) (2021)
- [Analysis and Optimisation of Bellman Residual Errors with Neural Function Approximation](https://mlsanity.com/paper/2106.08774/) (2021)
- [Memorization-Dilation: Modeling Neural Collapse Under Label Noise](https://mlsanity.com/paper/2206.05530/) (2022)
- [Transferring Learning Trajectories of Neural Networks](https://mlsanity.com/paper/2305.14122/) (2023)
- [Non-Adaptive Adversarial Face Generation](https://mlsanity.com/paper/2507.12107/) (2025)
- [Parameter Convex Neural Networks](https://mlsanity.com/paper/2206.05562/) (2022)
- [A Dynamics Theory of Implicit Regularization in Deep Low-Rank Matrix Factorization](https://mlsanity.com/paper/2212.14150/) (2022)
- [Learnable Game-theoretic Policy Optimization for Data-centric Self-explanation Rationalization](https://mlsanity.com/paper/2510.13393/) (2025)
- [Structural Correspondence and Universal Approximation in Diagonal plus Low-Rank Neural Networks](https://mlsanity.com/paper/2605.05659/) (2026)
- [Training Deep Neural Networks via Branch-and-Bound](https://mlsanity.com/paper/2104.01730/) (2021)
- [Complexity from Adaptive-Symmetries Breaking: Global Minima in the Statistical Mechanics of Deep Neural Networks](https://mlsanity.com/paper/2201.07934/) (2022)
- [Computing frustration and near-monotonicity in deep neural networks](https://mlsanity.com/paper/2510.05286/) (2025)

## Abstract (arXiv)

We study the error landscape of deep linear and nonlinear neural networks with the squared error loss. Minimizing the loss of a deep linear neural network is a nonconvex problem, and despite recent progress, our understanding of this loss surface is still incomplete. For deep linear networks, we present necessary and sufficient conditions for a critical point of the risk function to be a global minimum. Surprisingly, our conditions provide an efficiently checkable test for global optimality, while such tests are typically intractable in nonconvex optimization. We further extend these results to deep nonlinear neural networks and prove similar sufficient conditions for global optimality, albeit in a more limited function space setting.