mlsanityData to 14 Oct 2026

Federated Optimization:Distributed Optimization Beyond the Datacenter

arXiv 1511.03575 ↗First posted 11 Nov 2015cs.LG math.OCcited by 195

Not on the map. Why?

Abstract (arXiv)

We introduce a new and increasingly relevant setting for distributed optimization in machine learning, where the data defining the optimization are distributed (unevenly) over an extremely large number of nodes, but the goal remains to train a high-quality centralized model. We refer to this setting as Federated Optimization. In this setting, communication efficiency is of utmost importance. A motivating example for federated optimization arises when we keep the training data locally on users' mobile devices rather than logging it to a data center for training. Instead, the mobile devices are used as nodes performing computation on their local data in order to update a global model. We suppose that we have an extremely large number of devices in our network, each of which has only a tiny fraction of data available totally; in particular, we expect the number of data points available locally to be much smaller than the number of devices. Additionally, since different users generate data with different patterns, we assume that no device has a representative sample of the overall distribution. We show that existing algorithms are not suitable for this setting, and propose a new algorithm which shows encouraging experimental results. This work also sets a path for future research needed in the context of federated optimization.

Key facts. Cited by 195 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Nov 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 55; 2022: 48; 2023: 34; 2024: 26; 2025: 21; 2026: 11.

195 papers.

Sort:

By area:

  1. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 321
  2. 2021Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Modelscited by 108
  3. 2021Fast Federated Learning in the Presence of Arbitrary Device Unavailabilitycited by 72
  4. 2023TARGET: Federated Class-Continual Learning via Exemplar-Free Distillationcited by 65
  5. 2021Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advancescited by 54
  6. 2022Federated Learning via Inexact ADMMcited by 44
  7. 2021FedCor: Correlation-Based Active Client Selection Strategy for Heterogeneous Federated Learningcited by 43
  8. 2021Gradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrixcited by 43
  9. 2021Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspectivecited by 38
  10. 2021Scaling Federated Learning for Fine-tuning of Large Language Modelscited by 36
  11. 2021Non-IID data and Continual Learning processes in Federated Learning: A long road aheadcited by 36
  12. 2022Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customizationcited by 35
38 more
  1. 2021Concept drift detection and adaptation for federated and continual learningcited by 32
  2. 2021Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learningcited by 31
  3. 2021Ground-Assisted Federated Learning in LEO Satellite Constellationscited by 31
  4. 2021SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networkscited by 30
  5. 2021FEDZIP: A Compression Framework for Communication-Efficient Federated Learningcited by 27
  6. 2022FLUTE: A Scalable, Extensible Framework for High-Performance Federated Learning Simulationscited by 27
  7. 2022Reconstructing Training Data from Model Gradient, Provablycited by 27
  8. 2023DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learningcited by 24
  9. 2023GradientCoin: A Peer-to-Peer Decentralized Large Language Modelscited by 23
  10. 2021Federated Learning with Taskonomy for Non-IID Datacited by 21
  11. 2022Federated Learning with Heterogeneous Architectures using Graph HyperNetworkscited by 19
  12. 2021Communication-Efficient Federated Learning via Quantized Compressed Sensingcited by 18
  13. 2023Communication-Efficient Split Learning via Adaptive Feature-Wise Compressioncited by 18
  14. 2023FedHCA2: Towards Hetero-Client Federated Multi-Task Learningcited by 17
  15. 2021A Decentralized Federated Learning Framework via Committee Mechanism with Convergence Guaranteecited by 16
  16. 2021KKT Conditions, First-Order and Second-Order Optimization, and Distributed Optimization: Tutorial and Surveycited by 16
  17. 2022An Equivalence Between Data Poisoning and Byzantine Gradient Attackscited by 16
  18. 2022Defending against the Label-flipping Attack in Federated Learningcited by 16
  19. 2022FL-Defender: Combating Targeted Attacks in Federated Learningcited by 15
  20. 2021Communication-Efficient Federated Learning with Binary Neural Networkscited by 14
  21. 2021Communication-Efficient ADMM-based Federated Learningcited by 14
  22. 2022Anchor Sampling for Federated Learning with Partial Client Participationcited by 14
  23. 2023SimFBO: Towards Simple, Flexible and Communication-efficient Federated Bilevel Learningcited by 14
  24. 2024Unlearning during Learning: An Efficient Federated Machine Unlearning Methodcited by 14
  25. 2021Efficient Client Contribution Evaluation for Horizontal Federated Learningcited by 13
  26. 2021Federated Deep Learning with Bayesian Privacycited by 13
  27. 2023Communication-Efficient Federated Hypergradient Computation via Aggregated Iterative Differentiationcited by 13
  28. 2023DIGEST: Fast and Communication Efficient Decentralized Learning with Local Updatescited by 12
  29. 2021Differentially Private Federated Learning via Inexact ADMMcited by 11
  30. 2021Federated Feature Selection for Cyber-Physical Systems of Systemscited by 11
  31. 2021Tackling Dynamics in Federated Incremental Learning with Variational Embedding Rehearsalcited by 11
  32. 2021FedCostWAvg: A new averaging for better Federated Learningcited by 11
  33. 2024Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivitycited by 11
  34. 2021A Unified Transferable Model for ML-Enhanced DBMScited by 10
  35. 2021SoK: Machine Learning Governancecited by 10
  36. 2023Who Leaked the Model? Tracking IP Infringers in Accountable Federated Learningcited by 10
  37. 2022FedDKD: Federated Learning with Decentralized Knowledge Distillationcited by 9
  38. 2022Completely Heterogeneous Federated Learningcited by 9

50 most cited of 195 All 195 (JSON)