mlsanityData to 14 Oct 2026

Federated Optimization in Heterogeneous Networks

arXiv 1812.06127 ↗First posted 14 Dec 2018cs.LG stat.MLcited by 2,987

Not on the map. Why?

Cited by 69 in Knowledge distillation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Federated Learning is a distributed learning paradigm with two key challenges that differentiate it from traditional distributed optimization: (1) significant variability in terms of the systems characteristics on each device in the network (systems heterogeneity), and (2) non-identically distributed data across the network (statistical heterogeneity). In this work, we introduce a framework, FedProx, to tackle heterogeneity in federated networks. FedProx can be viewed as a generalization and re-parametrization of FedAvg, the current state-of-the-art method for federated learning. While this re-parameterization makes only minor modifications to the method itself, these modifications have important ramifications both in theory and in practice. Theoretically, we provide convergence guarantees for our framework when learning over data from non-identical distributions (statistical heterogeneity), and while adhering to device-level systems constraints by allowing each participating device to perform a variable amount of work (systems heterogeneity). Practically, we demonstrate that FedProx allows for more robust convergence than FedAvg across a suite of realistic federated datasets. In particular, in highly heterogeneous settings, FedProx demonstrates significantly more stable and accurate convergence behavior relative to FedAvg---improving absolute test accuracy by 22% on average.

Key facts. Cited by 2,987 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 295; 2022: 433; 2023: 541; 2024: 627; 2025: 590; 2026: 500.

2,987 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  2. 2021Model-Contrastive Federated Learningcited by 766
  3. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  4. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  5. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  6. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  7. 2021Towards Personalized Federated Learningcited by 464
  8. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  9. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
  10. 2021A Field Guide to Federated Optimizationcited by 315
  11. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
  12. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
38 more
  1. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
  2. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  3. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 192
  4. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 190
  5. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 185
  6. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 182
  7. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  8. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  9. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
  10. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 146
  11. 2022Generalized Federated Learning via Sharpness Aware Minimizationcited by 144
  12. 2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 142
  13. 2021FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learningcited by 141
  14. 2021Federated Learning for Open Bankingcited by 141
  15. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 136
  16. 2022PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Modelcited by 134
  17. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 128
  18. 2022FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settingscited by 125
  19. 2022FedALA: Adaptive Local Aggregation for Personalized Federated Learningcited by 122
  20. 2024Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Modelscited by 116
  21. 2021Decentralized Federated Averagingcited by 114
  22. 2021Federated Graph Classification over Non-IID Graphscited by 114
  23. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 113
  24. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
  25. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 109
  26. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 109
  27. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  28. 2021Distributionally Robust Federated Averagingcited by 104
  29. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  30. 2023Stochastic Controlled Averaging for Federated Learning with Communication Compressioncited by 101
  31. 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 99
  32. 2022Improving Generalization in Federated Learning by Seeking Flat Minimacited by 98
  33. 2022Neurotoxin: Durable Backdoors in Federated Learningcited by 98
  34. 2021Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applicationscited by 97
  35. 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
  36. 2021PPFL: Privacy-preserving Federated Learning with Trusted Execution Environmentscited by 93
  37. 2021Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learningcited by 93
  38. 2021Trustworthy AI: A Computational Perspectivecited by 93

50 most cited of 2,987 All 2,987 (JSON)