# On Convergence Lemma and Convergence Stability for Piecewise Analytic Functions

arXiv: https://arxiv.org/abs/2204.01643 · HTML: https://mlsanity.com/paper/2204.01643/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 40 references that are in the map.
- First posted on arXiv 4 Apr 2022.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [A Convergence Theory for Deep Learning via Over-Parameterization](https://mlsanity.com/paper/1811.03962/) (2018)
- [Gradient Descent Provably Optimizes Over-parameterized Neural Networks](https://mlsanity.com/paper/1810.02054/) (2018)
- [Gradient Descent Finds Global Minima of Deep Neural Networks](https://mlsanity.com/paper/1811.03804/) (2018)
- [Pooling Methods in Deep Neural Networks, a Review](https://mlsanity.com/paper/2009.07485/) (2020)

## Built on by (most cited first, 1 of 1)

- [A survey on algorithms for Nash equilibria in finite normal-form games](https://mlsanity.com/paper/2312.11063/) (2023)

## Abstract (arXiv)

In this work, a convergence lemma for function f being finite compositions of analytic mappings and the maximum operator is proved. The lemma shows that the set of δ-stationary points near an isolated local minimum point x^* is shrinking to x^* as δ→ 0. It is a natural extension of the version for strongly convex C1 functions. However, the correctness of the lemma is subtle. Analytic mappings are necessary for the lemma in the sense that replacing it with differentiable or C^∞ mappings makes the lemma false. The proof is based on stratification theorems of semi-analytic sets by Lojasiewicz. An extension of this proof presents a geometric characterization of the set of stationary points of f. Finally, a notion of stability on stationary points, called convergence stability, is proposed. It asks, under small numerical errors, whether a reasonable convergent optimization method started near a stationary point should eventually converge to the same stationary point. The concept of convergence stability becomes nontrivial qualitatively only when the objective function is both nonsmooth and nonconvex. Via the convergence lemma, an intuitive equivalent condition for convergence stability of f is proved. These results together provide a new geometric perspective to study the problem of "where-to-converge" in nonsmooth nonconvex optimization.