# Convergence Analysis of the Last Iterate in Distributed Stochastic Gradient Descent with Momentum

arXiv: https://arxiv.org/abs/2505.10889 · HTML: https://mlsanity.com/paper/2505.10889/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 May 2025.

## Built on by (most cited first, 1 of 1)

- [Adam under Generalized Smoothness with Second-Moment-Type Stochastic Gradients](https://mlsanity.com/paper/2609.37787/) (2026)

## Abstract (arXiv)

Distributed stochastic gradient methods are widely used to preserve data privacy and ensure scalability in large-scale learning tasks. While existing theory on distributed momentum Stochastic Gradient Descent (mSGD) mainly focuses on time-averaged convergence, the more practical last-iterate convergence remains underexplored. In this work, we analyze the last-iterate convergence behavior of distributed mSGD in non-convex settings under the classical Robbins-Monro step-size schedule. We prove both almost sure convergence and L2 convergence of the last iterate, and derive convergence rates. We further show that momentum can accelerate early-stage convergence, and provide experiments to support our theory.