mlsanityData to 7 Oct 2026

Tackling Computational Heterogeneity in FL: A Few Theoretical Insights

arXiv 2307.06283 ↗First posted 12 Jul 2023cs.LG cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

The future of machine learning lies in moving data collection along with training to the edge. Federated Learning, for short FL, has been recently proposed to achieve this goal. The principle of this approach is to aggregate models learned over a large number of distributed clients, i.e., resource-constrained mobile devices that collect data from their environment, to obtain a new more general model. The latter is subsequently redistributed to clients for further training. A key feature that distinguishes federated learning from data-center-based distributed training is the inherent heterogeneity. In this work, we introduce and analyse a novel aggregation framework that allows for formalizing and tackling computational heterogeneity in federated optimization, in terms of both heterogeneous data and local updates. Proposed aggregation algorithms are extensively analyzed from a theoretical, and an experimental prospective.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 39 of its 46 references that are in the map. First posted on arXiv 12 Jul 2023.

Builds on

39 of 46 references.

References · 39

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  5. 2020Adaptive Federated Optimizationcited by 1,036
  6. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  7. 2018How To Backdoor Federated Learningcited by 745
  8. 2021Towards Personalized Federated Learningcited by 462
  9. 2019Agnostic Federated Learningcited by 451
  10. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
  11. 2019Fair Resource Allocation in Federated Learningcited by 389
  12. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
27 more
  1. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 347
  2. 2019SGD: General Analysis and Improved Ratescited by 206
  3. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 196
  4. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  5. 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 154
  6. 2020Optimal Client Sampling for Federated Learningcited by 123
  7. 2021Local Learning Matters: Rethinking Data Heterogeneity in Federated Learningcited by 114
  8. 2019Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregationcited by 99
  9. 2022Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learningcited by 81
  10. 2019Accelerating Federated Learning via Momentum Gradient Descentcited by 75
  11. 2019Variational Federated Multi-Task Learningcited by 69
  12. 2022Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Featurescited by 63
  13. 2022FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learningcited by 62
  14. 2019Semi-Cyclic Stochastic Gradient Descentcited by 40
  15. 2020Stochastic Client Selection for Federated Learning with Volatile Clientscited by 38
  16. 2022Mechanisms that Incentivize Data Sharing in Federated Learningcited by 38
  17. 2021A Federated Learning Aggregation Algorithm for Pervasive Computing: Evaluation and Comparisoncited by 31
  18. 2019Federated Learning with Autotuned Communication-Efficient Secure Aggregationcited by 23
  19. 2021Personalized Federated Learning for Heterogeneous Clients with Clustered Knowledge Transfercited by 23
  20. 2022Accelerated Federated Learning with Decoupled Adaptive Optimizationcited by 23
  21. 2021FedPAGE: A Fast Local Stochastic Gradient Method for Communication-Efficient Federated Learningcited by 19
  22. 2022FedShuffle: Recipes for Better Use of Local Work in Federated Learningcited by 19
  23. 2022Federated Select: A Primitive for Communication- and Memory-Efficient Federated Learningcited by 11
  24. 2014High-Performance Distributed ML at Scale through Parameter Server Consistency Modelscited by 10
  25. 2022Personalized Federated Learning with Communication Compressioncited by 6
  26. 2019Limited-Memory BFGS with Displacement Aggregationcited by 4
  27. 2022On the Convergence of Heterogeneous Federated Learning with Arbitrary Adaptive Online Model Pruningcited by 4

Built on by

Not cited yet.