mlsanityData to 7 Oct 2026

Better Methods and Theory for Federated Learning: Compression, Client Selection and Heterogeneity

arXiv 2207.00392 ↗First posted 1 Jul 2022cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is an emerging machine learning paradigm involving multiple clients, e.g., mobile phone devices, with an incentive to collaborate in solving a machine learning problem coordinated by a central server. FL was proposed in 2016 by Konevcn'y et al. and McMahan et al. as a viable privacy-preserving alternative to traditional centralized machine learning since, by construction, the training data points are decentralized and never transferred by the clients to a central server. Therefore, to a certain degree, FL mitigates the privacy risks associated with centralized data collection. Unfortunately, optimization for FL faces several specific issues that centralized optimization usually does not need to handle. In this thesis, we identify several of these challenges and propose new methods and algorithms to address them, with the ultimate goal of enabling practical FL solutions supported with mathematically rigorous guarantees.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 44 of its 285 references that are in the map. First posted on arXiv 1 Jul 2022.

Builds on

44 of 285 references.

References · 44

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  3. 2015Prioritized Experience Replaycited by 1,070
  4. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  5. 2018LEAF: A Benchmark for Federated Settingscited by 780
  6. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  7. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
  8. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  9. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
  10. 2021A Field Guide to Federated Optimizationcited by 312
  11. 2020Federated Learning of a Mixture of Global and Local Modelscited by 208
  12. 2017Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryptioncited by 174
32 more
  1. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 162
  2. 2019On the Convergence of Local Descent Methods in Federated Learningcited by 150
  3. 2020On Biased Compression for Distributed Learningcited by 140
  4. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  5. 2020Optimal Client Sampling for Federated Learningcited by 123
  6. 2015Online Batch Selection for Faster Training of Neural Networkscited by 121
  7. 2019Federated Learning for Emoji Prediction in a Mobile Keyboardcited by 115
  8. 2019Distributed Learning with Compressed Gradient Differencescited by 110
  9. 2019Natural Compression for Distributed Deep Learningcited by 88
  10. 2019Unified Optimal Analysis of the (Stochastic) Gradient Methodcited by 81
  11. 2019First Analysis of Local GD on Heterogeneous Datacited by 80
  12. 2019Variance Reduced Local SGD with Lower Communication Complexitycited by 80
  13. 2020On the Convergence of SGD with Biased Gradientscited by 79
  14. 2019Stochastic Distributed Learning with Gradient Quantization and Variance Reductioncited by 64
  15. 2020Federated Learning With Quantized Global Model Updatescited by 54
  16. 2018Distributed learning with compressed gradientscited by 53
  17. 2016AIDE: Fast and Communication Efficient Distributed Optimizationcited by 42
  18. 2016Unified Convergence Analysis of Stochastic Momentum Methods for Convex and Non-convex Optimizationcited by 34
  19. 2021Personalized Federated Learning: A Unified Framework and Universal Optimization Techniquescited by 34
  20. 2021Proximal and Federated Random Reshufflingcited by 33
  21. 2019Hybrid Stochastic Gradient Descent Algorithms for Stochastic Nonconvex Optimizationcited by 31
  22. 2019On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learningcited by 31
  23. 2019Local AdaAlter: Communication-Efficient Stochastic Gradient Descent with Adaptive Learning Ratescited by 26
  24. 2019Communication-Efficient Local Decentralized SGD Methodscited by 21
  25. 2022Federated Learning Challenges and Opportunities: An Outlookcited by 19
  26. 2022FedShuffle: Recipes for Better Use of Local Work in Federated Learningcited by 19
  27. 2021EF21 with Bells & Whistles: Six Algorithmic Extensions of Modern Error Feedbackcited by 6
  28. 2020On Tight Convergence Rates of Without-replacement SGDcited by 3
  29. 2015Primal Method for ERM with Flexible Mini-batching Schemes and Non-convex Lossescited by 2
  30. 2019Global Momentum Compression for Sparse Communication in Distributed Learningcited by 2
  31. 2021A first look into the carbon footprint of federated learningcited by 2
  32. 2018k-SVRG: Variance Reduction for Large Scale Optimizationcited by 1

Built on by

Not cited yet.