mlsanityData to 7 Oct 2026

Federated Learning with Superquantile Aggregation for Heterogeneous Data

arXiv 2112.09429 ↗First posted 17 Dec 2021cs.LG math.OC stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

We present a federated learning framework that is designed to robustly deliver good predictive performance across individual clients with heterogeneous data. The proposed approach hinges upon a superquantile-based learning objective that captures the tail statistics of the error distribution over heterogeneous clients. We present a stochastic training algorithm that interleaves differentially private client filtering with federated averaging steps. We prove finite time convergence guarantees for the algorithm: O(1/√T) in the nonconvex case in T communication rounds and O(exp(-T/kappa3/2) + kappa/T) in the strongly convex case with local condition number kappa. Experimental results on benchmark datasets for federated learning demonstrate that our approach is competitive with classical ones in terms of average error and outperforms them in terms of tail statistics of the error.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 67 of its 129 references that are in the map. First posted on arXiv 17 Dec 2021.

Builds on

67 of 129 references.

References · 67

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  6. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  7. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  8. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  9. 2020Adaptive Federated Optimizationcited by 1,036
  10. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  11. 2019Towards Federated Learning at Scale: System Designcited by 826
  12. 2018LEAF: A Benchmark for Federated Settingscited by 780
38 more
  1. 2017Federated Multi-Task Learningcited by 669
  2. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  3. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  4. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  5. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  6. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  7. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  8. 2019Agnostic Federated Learningcited by 451
  9. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  10. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
  11. 2019Fair Resource Allocation in Federated Learningcited by 389
  12. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  13. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 361
  14. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
  15. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  16. 2019Robust Aggregation for Federated Learningcited by 314
  17. 2021A Field Guide to Federated Optimizationcited by 312
  18. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  19. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  20. 2017EMNIST: an extension of MNIST to handwritten letterscited by 212
  21. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 202
  22. 2020Large-Scale Methods for Distributionally Robust Optimizationcited by 179
  23. 2019Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learningcited by 169
  24. 2015Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approachcited by 162
  25. 2018Stochastic model-based minimization of weakly convex functionscited by 157
  26. 2016Variance-based regularization with convex objectivescited by 153
  27. 2020The Discrete Gaussian for Differential Privacycited by 134
  28. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
  29. 2019Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronizationcited by 106
  30. 2021Distributionally Robust Federated Averagingcited by 104
  31. 2020UVeQFed: Universal Vector Quantization for Federated Learningcited by 103
  32. 2020Tilted Empirical Risk Minimizationcited by 103
  33. 2020Robust Federated Learning: The Case of Affine Distribution Shiftscited by 102
  34. 2017On the convergence properties of a K-step averaging stochastic gradient descent algorithm for nonconvex optimizationcited by 91
  35. 2019Fairness risk measurescited by 85
  36. 2021Federated Evaluation and Tuning for On-Device Personalization: System Design & Applicationscited by 84
  37. 2015Policy Gradient for Coherent Risk Measurescited by 74
  38. 2020Over-the-Air Federated Learning from Heterogeneous Datacited by 72

50 most cited of 67 All 67 (JSON)

Built on by

Not cited yet.