mlsanityData to 7 Oct 2026

Review of Mathematical Optimization in Federated Learning

arXiv 2412.01630 ↗First posted 2 Dec 2024cs.LG cs.DCcited by 3

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) has been becoming a popular interdisciplinary research area in both applied mathematics and information sciences. Mathematically, FL aims to collaboratively optimize aggregate objective functions over distributed datasets while satisfying a variety of privacy and system constraints.Different from conventional distributed optimization methods, FL needs to address several specific issues (e.g., non-i.i.d. data distributions and differential private noises), which pose a set of new challenges in the problem formulation, algorithm design, and convergence analysis. In this paper, we will systematically review existing FL optimization research including their assumptions, formulations, methods, and theoretical results. Potential future directions are also discussed.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 156 of its 233 references that are in the map. First posted on arXiv 2 Dec 2024.

Builds on

156 of 233 references.

References · 156

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2016Deep Learning with Differential Privacycited by 2,682
  5. 2021Convex optimizationcited by 2,479
  6. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  7. 2020Adaptive Federated Optimizationcited by 1,036
  8. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  9. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  10. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  11. 2019Introduction to Online Convex Optimizationcited by 793
  12. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 698
38 more
  1. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  2. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  3. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  4. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  5. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  6. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  7. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  8. 2018The Hidden Vulnerability of Distributed Learning in Byzantiumcited by 324
  9. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  10. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
  11. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  12. 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 253
  13. 2017Sparse Communication for Distributed Gradient Descentcited by 243
  14. 2018Online Learning: A Comprehensive Surveycited by 203
  15. 2017Gradient Sparsification for Communication-Efficient Distributed Optimizationcited by 199
  16. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 196
  17. 2018AdaGrad stepsizes: Sharp convergence over nonconvex landscapescited by 182
  18. 2021Federated Learning for Smart Healthcare: A Surveycited by 182
  19. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  20. 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 175
  21. 2020A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learningcited by 160
  22. 2013On the Convergence of Decentralized Gradient Descentcited by 158
  23. 2020Federated Continual Learning with Weighted Inter-client Transfercited by 158
  24. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 157
  25. 2017A Survey on Homomorphic Encryption Schemes: Theory and Implementationcited by 154
  26. 2020On Biased Compression for Distributed Learningcited by 140
  27. 2020Understanding Gradient Clipping in Private SGD: A Geometric Perspectivecited by 140
  28. 2020Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challengescited by 138
  29. 2020Learning from History for Byzantine Robust Optimizationcited by 137
  30. 2016Distributed Mean Estimation with Limited Communicationcited by 134
  31. 2020Byzantine-Robust Learning on Heterogeneous Datasets via Bucketingcited by 125
  32. 2019SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentumcited by 121
  33. 2019Overcoming Forgetting in Federated Learning on Non-IID Datacited by 120
  34. 2018Distributed Differential Privacy via Shufflingcited by 115
  35. 2021Decentralized Federated Averagingcited by 114
  36. 2018Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithmscited by 106
  37. 2019Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronizationcited by 106
  38. 2022Vertical Federated Learning: Concepts, Advances and Challengescited by 104

50 most cited of 156 All 156 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Federated Learning from Molecules to Processes: A Perspectivecited by 2
  2. 2025Objective Value Change and Shape-Based Accelerated Optimization for the Neural Network Approximationcited by 0
  3. 2026A Momentum-Based Variance-Reduced Algorithm for Federated Multiobjective Optimizationcited by 0