# Every Local Minimum Value is the Global Minimum Value of Induced Model in Non-convex Machine Learning

arXiv: https://arxiv.org/abs/1904.03673 · HTML: https://mlsanity.com/paper/1904.03673/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2019.

## Built on by (most cited first, 9 of 9)

- [Full Parameter Fine-tuning for Large Language Models with Limited Resources](https://mlsanity.com/paper/2306.09782/) (2023)
- [Rethinking Model Ensemble in Transfer-based Adversarial Attacks](https://mlsanity.com/paper/2303.09105/) (2023)
- [GEMEL: Model Merging for Memory-Efficient, Real-Time Video Analytics at the Edge](https://mlsanity.com/paper/2201.07705/) (2022)
- [Laziness, Barren Plateau, and Noise in Machine Learning](https://mlsanity.com/paper/2206.09313/) (2022)
- [On Quantum Speedups for Nonconvex Optimization via Quantum Tunneling Walks](https://mlsanity.com/paper/2209.14501/) (2022)
- [Zeroth-Order Negative Curvature Finding: Escaping Saddle Points without Gradients](https://mlsanity.com/paper/2210.01496/) (2022)
- [Exploring Ordinality in Text Classification: A Comparative Study of Explicit and Implicit Techniques](https://mlsanity.com/paper/2405.11775/) (2024)
- [Enhancing Adversarial Attacks: The Similar Target Method](https://mlsanity.com/paper/2308.10743/) (2023)
- [Algebraic Approach to Ridge-Regularized Mean Squared Error Minimization in Minimal ReLU Neural Network](https://mlsanity.com/paper/2508.17783/) (2025)

## Abstract (arXiv)

For nonconvex optimization in machine learning, this article proves that every local minimum achieves the globally optimal value of the perturbable gradient basis model at any differentiable point. As a result, nonconvex machine learning is theoretically as supported as convex machine learning with a handcrafted basis in terms of the loss at differentiable local minima, except in the case when a preference is given to the handcrafted basis over the perturbable gradient basis. The proofs of these results are derived under mild assumptions. Accordingly, the proven results are directly applicable to many machine learning models, including practical deep neural networks, without any modification of practical methods. Furthermore, as special cases of our general results, this article improves or complements several state-of-the-art theoretical results on deep neural networks, deep residual networks, and overparameterized deep neural networks with a unified proof technique and novel geometric insights. A special case of our results also contributes to the theoretical foundation of representation learning.