# A KL-based Analysis Framework with Applications to Non-Descent Optimization Methods

arXiv: https://arxiv.org/abs/2406.02273 · HTML: https://mlsanity.com/paper/2406.02273/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 40 references that are in the map.
- First posted on arXiv 4 Jun 2024.

## Builds on (most cited first, 6 of 6)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Minibatch vs Local SGD with Shuffling: Tight Convergence Bounds and Beyond](https://mlsanity.com/paper/2110.10342/) (2021)
- [Central Server Free Federated Learning over Single-sided Trust Social Networks](https://mlsanity.com/paper/1910.04956/) (2019)
- [A Normal Map-Based Proximal Stochastic Gradient Method: Convergence and Identification Properties](https://mlsanity.com/paper/2305.05828/) (2023)
- [Convergence of SGD with momentum in the nonconvex case: A time window-based analysis](https://mlsanity.com/paper/2405.16954/) (2024)

## Built on by (most cited first, 1 of 1)

- [Enhancing Convergence of Decentralized Gradient Tracking under the KL Property](https://mlsanity.com/paper/2412.09556/) (2024)

## Abstract (arXiv)

We propose a novel analysis framework for non-descent-type optimization methodologies in nonconvex scenarios based on the Kurdyka-Lojasiewicz property. Our framework allows covering a broad class of algorithms, including those commonly employed in stochastic and distributed optimization. Specifically, it enables the analysis of first-order methods that lack a sufficient descent property and do not require access to full (deterministic) gradient information. We leverage this framework to establish, for the first time, iterate convergence and the corresponding rates for the decentralized gradient method and federated averaging under mild assumptions. Furthermore, based on the new analysis techniques, we show the convergence of the random reshuffling and stochastic gradient descent method without necessitating typical a priori bounded iterates assumptions.