# Enhance Local Consistency in Federated Learning: A Multi-Step Inertial Momentum Approach

arXiv: https://arxiv.org/abs/2302.05726 · HTML: https://mlsanity.com/paper/2302.05726/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2023.

## Built on by (most cited first, 11 of 11)

- [FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy](https://mlsanity.com/paper/2302.10429/) (2023)
- [Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initialization](https://mlsanity.com/paper/2306.05706/) (2023)
- [Accelerating Heterogeneous Federated Learning with Closed-form Classifiers](https://mlsanity.com/paper/2406.01116/) (2024)
- [Federated Learning with Manifold Regularization and Normalized Update Reaggregation](https://mlsanity.com/paper/2311.05924/) (2023)
- [Window-based Model Averaging Improves Generalization in Heterogeneous Federated Learning](https://mlsanity.com/paper/2310.01366/) (2023)
- [FilFL: Client Filtering for Optimized Client Participation in Federated Learning](https://mlsanity.com/paper/2302.06599/) (2023)
- [When Foresight Pruning Meets Zeroth-Order Optimization: Efficient Federated Learning for Low-Memory Devices](https://mlsanity.com/paper/2405.04765/) (2024)
- [A-FedPD: Aligning Dual-Drift is All Federated Primal-Dual Learning Needs](https://mlsanity.com/paper/2409.18915/) (2024)
- [Convergent Differential Privacy Analysis for General Federated Learning](https://mlsanity.com/paper/2408.15621/) (2024)
- [On the Limits of Momentum in Decentralized and Federated Optimization](https://mlsanity.com/paper/2511.20168/) (2025)
- [Rethinking the Personalized Relaxed Initialization in the Federated Learning: Consistency and Generalization](https://mlsanity.com/paper/2604.12768/) (2026)

## Abstract (arXiv)

Federated learning (FL), as a collaborative distributed training paradigm with several edge computing devices under the coordination of a centralized server, is plagued by inconsistent local stationary points due to the heterogeneity of the local partial participation clients, which precipitates the local client-drifts problems and sparks off the unstable and slow convergence, especially on the aggravated heterogeneous dataset. To address these issues, we propose a novel federated learning algorithm, named FedMIM, which adopts the multi-step inertial momentum on the edge devices and enhances the local consistency for free during the training to improve the robustness of the heterogeneity. Specifically, we incorporate the weighted global gradient estimations as the inertial correction terms to guide both the local iterates and stochastic gradient estimation, which can reckon the global objective optimization on the edges' heterogeneous dataset naturally and maintain the demanding consistent iteration locally. Theoretically, we show that FedMIM achieves the O(frac1√SKT) convergence rate with a linear speedup property with respect to the number of selected clients S and proper local interval K in communication round T without convex assumption. Empirically, we conduct comprehensive experiments on various real-world datasets and demonstrate the efficacy of the proposed FedMIM against several state-of-the-art baselines.