mlsanityData to 7 Oct 2026

Scaff-PD: Communication Efficient Fair and Robust Federated Learning

arXiv 2307.13381 ↗First posted 25 Jul 2023cs.LG cs.DC math.OCcited by 6

Not on the map. Why?

Abstract (arXiv)

We present Scaff-PD, a fast and communication-efficient algorithm for distributionally robust federated learning. Our approach improves fairness by optimizing a family of distributionally robust objectives tailored to heterogeneous clients. We leverage the special structure of these objectives, and design an accelerated primal dual (APD) algorithm which uses bias corrected local steps (as in Scaffold) to achieve significant gains in communication efficiency and convergence speed. We evaluate Scaff-PD on several benchmark datasets and demonstrate its effectiveness in improving fairness and robustness while maintaining competitive accuracy. Our results suggest that Scaff-PD is a promising approach for federated learning in resource-constrained and heterogeneous settings.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 39 of its 65 references that are in the map. First posted on arXiv 25 Jul 2023.

Builds on

39 of 65 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2020Adaptive Federated Optimizationcited by 1,036
  6. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  7. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  8. 2019Agnostic Federated Learningcited by 451
  9. 2019Fair Resource Allocation in Federated Learningcited by 389
  10. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
  11. 2021A Field Guide to Federated Optimizationcited by 312
  12. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 303
27 more
  1. 2015Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulationscited by 285
  2. 2016Distributionally Robust Stochastic Optimization with Wasserstein Distancecited by 182
  3. 2020Large-Scale Methods for Distributionally Robust Optimizationcited by 179
  4. 2020BREEDS: Benchmarks for Subpopulation Shiftcited by 155
  5. 2020Minibatch vs Local SGD for Heterogeneous Distributed Learningcited by 147
  6. 2022On Feature Learning in the Presence of Spurious Correlationscited by 145
  7. 2020Salvaging Federated Learning by Local Adaptationcited by 142
  8. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  9. 2021Distributionally Robust Federated Averagingcited by 104
  10. 2020Tilted Empirical Risk Minimizationcited by 103
  11. 2020Collaborative Machine Learning with Incentive-Aware Model Rewardscited by 71
  12. 2020Distributionally Robust Losses for Latent Covariate Mixturescited by 57
  13. 2020Mitigating Bias in Federated Learningcited by 56
  14. 2021One for One, or All for All: Equilibria and Optimality of Collaboration in Federated Learningcited by 39
  15. 2022Mechanisms that Incentivize Data Sharing in Federated Learningcited by 38
  16. 2014Stochastic Primal-Dual Coordinate Method for Regularized Empirical Risk Minimizationcited by 30
  17. 2022TCT: Convexifying Federated Learning using Bootstrapped Neural Tangent Kernelscited by 26
  18. 2021Decentralized Local Stochastic Extra-Gradient for Variational Inequalitiescited by 25
  19. 2021Focus on the Common Good: Group Distributional Robustness Followscited by 25
  20. 2022SAPD+: An Accelerated Stochastic Method for Nonconvex-Concave Minimax Problemscited by 23
  21. 2020Device Heterogeneity in Federated Learning: A Superquantile Approachcited by 20
  22. 2021Efficient Algorithms for Federated Saddle Point Optimizationcited by 20
  23. 2022Proportional Fairness in Federated Learningcited by 20
  24. 2021Communication-Efficient Agnostic Federated Averagingcited by 10
  25. 2021Robust Accelerated Primal-Dual Methods for Computing Saddle Pointscited by 10
  26. 2022Communication-Efficient Distributionally Robust Decentralized Learningcited by 10
  27. 2022On the effectiveness of partial variance reduction in federated learning with heterogeneous datacited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 2.

6 papers.

  1. 2024Communication-Efficient Federated Group Distributionally Robust Optimizationcited by 3
  2. 2025Distributionally Robust Federated Learning with Client Drift Minimizationcited by 1
  3. 2025Distributionally Robust Federated Learning with Outlier Resiliencecited by 1
  4. 2024Federated Learning with Relative Fairnesscited by 0
  5. 2026FedeRage: Provably Convergent Agnostic Federated Learning under General Client Driftcited by 0
  6. 2026Communication-Efficient Agnostic Federated Learning via Faster Convergence and Compressioncited by 0