mlsanityData to 7 Oct 2026

FedLion: Faster Adaptive Federated Optimization with Fewer Communication

arXiv 2402.09941 ↗First posted 15 Feb 2024cs.LG cs.AI stat.MLcited by 6

Not on the map. Why?

Abstract (arXiv)

In Federated Learning (FL), a framework to train machine learning models across distributed data, well-known algorithms like FedAvg tend to have slow convergence rates, resulting in high communication costs during training. To address this challenge, we introduce FedLion, an adaptive federated optimization algorithm that seamlessly incorporates key elements from the recently proposed centralized adaptive algorithm, Lion (Chen et al. 2o23), into the FL framework. Through comprehensive evaluations on two widely adopted FL benchmarks, we demonstrate that FedLion outperforms previous state-of-the-art adaptive algorithms, including FAFED (Wu et al. 2023) and FedDA. Moreover, thanks to the use of signed gradients in local training, FedLion substantially reduces data transmission requirements during uplink communication when compared to existing adaptive algorithms, further reducing communication costs. Last but not least, this work also includes a novel theoretical analysis, showcasing that FedLion attains faster convergence rate than established FL algorithms like FedAvg.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 17 of its 20 references that are in the map. First posted on arXiv 15 Feb 2024.

Builds on

17 of 20 references.

References · 17

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2020Adaptive Federated Optimizationcited by 1,036
  4. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  5. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  6. 2023Symbolic Discovery of Optimization Algorithmscited by 423
  7. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
  8. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  9. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 143
  10. 2019Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronizationcited by 106
  11. 2019Accelerating Federated Learning via Momentum Gradient Descentcited by 75
  12. 2022Communication-Efficient Adaptive Federated Learningcited by 55
5 more
  1. 2019Stochastic Sign Descent Methods: New Algorithms and Better Theorycited by 49
  2. 2020Faster Non-Convex Federated Learning via Global and Local Momentumcited by 48
  3. 2021STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learningcited by 43
  4. 2022Faster Adaptive Federated Learningcited by 35
  5. 2022Accelerated Federated Learning with Decoupled Adaptive Optimizationcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 2.

6 papers.

  1. 2025Convergence Analysis of the Lion Optimizer in Centralized and Distributed Settingscited by 6
  2. 2025An Efficient Subspace Algorithm for Federated Learning on Heterogeneous Datacited by 4
  3. 2026Rethinking LoRA for Data Heterogeneous Federated Learning: Subspace and State Alignmentcited by 2
  4. 2024Deep Representation Learning for Multi-functional Degradation Modeling of Community-dwelling Aging Populationcited by 1
  5. 2024Distributed Sign Momentum with Local Steps for Training Transformerscited by 1
  6. 2026FedZMG: Efficient Client-Side Optimization in Federated Learningcited by 0