mlsanityData to 7 Oct 2026

Explicit Personalization and Local Training: Double Communication Acceleration in Federated Learning

arXiv 2305.13170 ↗First posted 22 May 2023cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Federated Learning is an evolving machine learning paradigm, in which multiple clients perform computations based on their individual private data, interspersed by communication with a remote server. A common strategy to curtail communication costs is Local Training, which consists in performing multiple local stochastic gradient descent steps between successive communication rounds. However, the conventional approach to local training overlooks the practical necessity for client-specific personalization, a technique to tailor local models to individual needs. We introduce Scafflix, a novel algorithm that efficiently integrates explicit personalization with local training. This innovative approach benefits from these two techniques, thereby achieving doubly accelerated communication, as we demonstrate both in theory and practice.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 37 of its 50 references that are in the map. First posted on arXiv 22 May 2023.

Builds on

37 of 50 references.

References · 37

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  6. 2018LEAF: A Benchmark for Federated Settingscited by 780
  7. 2021Model-Contrastive Federated Learningcited by 762
  8. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  9. 2019Federated Learning with Personalization Layerscited by 533
  10. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  11. 2021A Field Guide to Federated Optimizationcited by 312
  12. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
25 more
  1. 2020Federated Learning of a Mixture of Global and Local Modelscited by 208
  2. 2019SGD: General Analysis and Improved Ratescited by 206
  3. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  4. 2019FedHealth: A Federated Transfer Learning Framework for Wearable Healthcarecited by 182
  5. 2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 174
  6. 2019On the Convergence of Local Descent Methods in Federated Learningcited by 150
  7. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  8. 2021Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradientscited by 122
  9. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  10. 2020Local SGD: Unified Theory and New Efficient Methodscited by 88
  11. 2019A Unified Theory of SGD: Variance Reduction, Sampling, Quantization and Coordinate Descentcited by 84
  12. 2019First Analysis of Local GD on Heterogeneous Datacited by 80
  13. 2020From Local SGD to Local Fixed-Point Methods for Federated Learningcited by 74
  14. 2022Personalized Subgraph Federated Learningcited by 54
  15. 2023Efficient Personalized Federated Learning via Sparse Model-Adaptationcited by 38
  16. 2023Personalized Federated Learning under Mixture of Distributionscited by 37
  17. 2022Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learningcited by 33
  18. 2021FedJAX: Federated learning simulation with JAXcited by 32
  19. 2019Federated User Representation Learningcited by 31
  20. 2022RandProx: Primal-Dual Optimization Algorithms with Randomized Proximal Updatescited by 27
  21. 2022Can 5th Generation Local Training Methods Support Client Sampling? Yes!cited by 22
  22. 2022GradSkip: Communication-Accelerated Local Gradient Methods with Better Computational Complexitycited by 17
  23. 2015SparkNet: Training Deep Networks in Sparkcited by 13
  24. 2021FLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learningcited by 12
  25. 2022CompressedScaffnew: The First Theoretical Double Acceleration of Communication from Local Training and Compression in Distributed Optimizationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 0; 2026: 1.

5 papers.

  1. 2024FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneitycited by 13
  2. 2023Towards a Better Theoretical Understanding of Independent Subnetwork Trainingcited by 6
  3. 2024FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Modelscited by 5
  4. 2024Communication-Efficient Federated Group Distributionally Robust Optimizationcited by 3
  5. 2026BiCoLoR: Communication-Efficient Optimization with Bidirectional Compression and Local Trainingcited by 1