# On the Stability Analysis of Open Federated Learning Systems

arXiv: https://arxiv.org/abs/2209.12307 · HTML: https://mlsanity.com/paper/2209.12307/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 43 references that are in the map.
- First posted on arXiv 25 Sep 2022.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Local SGD Converges Fast and Communicates Little](https://mlsanity.com/paper/1805.09767/) (2018)
- [Train faster, generalize better: Stability of stochastic gradient descent](https://mlsanity.com/paper/1509.01240/) (2015)
- [FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization](https://mlsanity.com/paper/1909.13014/) (2019)
- [Tighter Theory for Local SGD on Identical and Heterogeneous Data](https://mlsanity.com/paper/1909.04746/) (2019)
- [On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization](https://mlsanity.com/paper/1905.03817/) (2019)
- [SCAFFOLD: Stochastic Controlled Averaging for Federated Learning](https://mlsanity.com/paper/1910.06378/) (2019)
- [Is Local SGD Better than Minibatch SGD?](https://mlsanity.com/paper/2002.07839/) (2020)
- [Protection Against Reconstruction and Its Applications in Private Federated Learning](https://mlsanity.com/paper/1812.00984/) (2018)
- [Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients](https://mlsanity.com/paper/2102.07053/) (2021)
- [SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum](https://mlsanity.com/paper/1910.00643/) (2019)
- [Local Adaptivity in Federated Learning: Convergence and Consistency](https://mlsanity.com/paper/2106.02305/) (2021)
- [Local AdaAlter: Communication-Efficient Stochastic Gradient Descent with Adaptive Learning Rates](https://mlsanity.com/paper/1911.09030/) (2019)
- [Multi-Agent Online Optimization with Delays: Asynchronicity, Adaptivity, and Optimism](https://mlsanity.com/paper/2012.11579/) (2020)
- [The Error-Feedback Framework: Better Rates for SGD with Delayed Gradients and Compressed Communication](https://mlsanity.com/paper/1909.05350/) (2019)
- [Local SGD With a Communication Overhead Depending Only on the Number of Workers](https://mlsanity.com/paper/2006.02582/) (2020)
- [CADA: Communication-Adaptive Distributed Adam](https://mlsanity.com/paper/2012.15469/) (2020)
- [Stability of Decentralized Gradient Descent in Open Multi-Agent Systems](https://mlsanity.com/paper/2009.05445/) (2020)
- [Open Multi-Agent Systems: Gossiping with Random Arrivals and Departures](https://mlsanity.com/paper/1709.05142/) (2017)
- [On Second-Moment Stability of Discrete-Time Linear Systems with General Stochastic Dynamics](https://mlsanity.com/paper/1911.00308/) (2019)
- [Optimization in Open Networks via Dual Averaging](https://mlsanity.com/paper/2105.13348/) (2021)
- [Equivalent Stability Notions, Lyapunov Inequality, and Its Application in Discrete-Time Linear Systems with Stochastic Dynamics Determined by an i.i.d. Process](https://mlsanity.com/paper/1902.11200/) (2019)

## Abstract (arXiv)

We consider the open federated learning (FL) systems, where clients may join and/or leave the system during the FL process. Given the variability of the number of present clients, convergence to a fixed model cannot be guaranteed in open systems. Instead, we resort to a new performance metric that we term the stability of open FL systems, which quantifies the magnitude of the learned model in open systems. Under the assumption that local clients' functions are strongly convex and smooth, we theoretically quantify the radius of stability for two FL algorithms, namely local SGD and local Adam. We observe that this radius relies on several key parameters, including the function condition number as well as the variance of the stochastic gradient. Our theoretical results are further verified by numerical simulations on both synthetic and real-world benchmark data-sets.