mlsanityData to 13 Oct 2026

FedAdaVR: Adaptive Variance Reduction for Robust Federated Learning under Limited Client Participation

arXiv 2601.22204 ↗First posted 29 Jan 2026cs.LG cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) encounters substantial challenges due to heterogeneity, leading to gradient noise, client drift, and partial client participation errors, the last of which is the most pervasive but remains insufficiently addressed in current literature. In this paper, we propose FedAdaVR, a novel FL algorithm aimed at solving heterogeneity issues caused by sporadic client participation by incorporating an adaptive optimiser with a variance reduction technique. This method takes advantage of the most recent stored updates from clients, even when they are absent from the current training round, thereby emulating their presence. Furthermore, we propose FedAdaVR-Quant, which stores client updates in quantised form, significantly reducing the memory requirements (by 50%, 75%, and 87.5%) of FedAdaVR while maintaining highly competitive model performance. We analyse the convergence behaviour of FedAdaVR under general nonconvex conditions and prove that our proposed algorithm can eliminate partial client participation error. Extensive experiments conducted on multiple datasets, under both independent and identically distributed (IID) and non-IID settings, demonstrate that FedAdaVR consistently outperforms state-of-the-art baseline methods.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 45 of its 64 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

45 of 64 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,653
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,426
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,786
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,328
  7. 2014Going Deeper with Convolutionscited by 4,239
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,836
  9. 2022A ConvNet for the 2020scited by 3,563
  10. 2018Federated Optimization in Heterogeneous Networkscited by 2,981
  11. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,352
  12. 2019Searching for MobileNetV3cited by 1,990
33 more
  1. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  2. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,227
  3. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,058
  4. 2020Adaptive Federated Optimizationcited by 1,041
  5. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 941
  6. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 903
  7. 2019On the Convergence of Adam and Beyondcited by 837
  8. 2018LEAF: A Benchmark for Federated Settingscited by 789
  9. 2019Large Batch Optimization for Deep Learning: Training BERT in 76 minutescited by 673
  10. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  11. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 565
  12. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 518
  13. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 453
  14. 2014SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectivescited by 401
  15. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 356
  16. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 304
  17. 2020AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradientscited by 283
  18. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  19. 2017SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradientcited by 220
  20. 2013Minimizing Finite Sums with the Stochastic Average Gradientcited by 164
  21. 2023Stochastic Controlled Averaging for Federated Learning with Communication Compressioncited by 100
  22. 2018Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGDcited by 77
  23. 2021Fast Federated Learning in the Presence of Arbitrary Device Unavailabilitycited by 72
  24. 2022FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learningcited by 64
  25. 2020Decentralized Federated Learning via Mutual Knowledge Transfercited by 43
  26. 2021Toward Communication Efficient Adaptive Gradient Methodcited by 25
  27. 2022DELTA: Diverse Client Sampling for Fasting Federated Learningcited by 24
  28. 2022AdaBest: Minimizing Client Drift in Federated Learning via Adaptive Bias Estimationcited by 21
  29. 2022Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devicescited by 12
  30. 2025A Convergence Analysis of Adaptive Optimizers under Floating-point Quantizationcited by 8
  31. 2023Federated Learning in the Presence of Adversarial Client Unavailabilitycited by 5
  32. 2023FedBug: A Bottom-Up Gradual Unfreezing Framework for Federated Learningcited by 3
  33. 2023Finite-sum optimization: Adaptivity to smoothness and loopless variance reductioncited by 1

Built on by

Not cited yet.