mlsanityData to 7 Oct 2026

On the Impact of Device and Behavioral Heterogeneity in Federated Learning

arXiv 2102.07500 ↗First posted 15 Feb 2021cs.LG cs.DC cs.PFcited by 4

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is becoming a popular paradigm for collaborative learning over distributed, private datasets owned by non-trusting entities. FL has seen successful deployment in production environments, and it has been adopted in services such as virtual keyboards, auto-completion, item recommendation, and several IoT applications. However, FL comes with the challenge of performing training over largely heterogeneous datasets, devices, and networks that are out of the control of the centralized FL server. Motivated by this inherent setting, we make a first step towards characterizing the impact of device and behavioral heterogeneity on the trained model. We conduct an extensive empirical study spanning close to 1.5K unique configurations on five popular FL benchmarks. Our analysis shows that these sources of heterogeneity have a major impact on both model performance and fairness, thus sheds light on the importance of considering heterogeneity in FL system design.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 31 of its 61 references that are in the map. First posted on arXiv 15 Feb 2021.

Builds on

31 of 61 references.

References · 31

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  7. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  8. 2019Towards Federated Learning at Scale: System Designcited by 826
  9. 2018LEAF: A Benchmark for Federated Settingscited by 780
  10. 2018How To Backdoor Federated Learningcited by 745
  11. 2017Federated Multi-Task Learningcited by 669
  12. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
19 more
  1. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
  2. 2019Agnostic Federated Learningcited by 451
  3. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  4. 2019Fair Resource Allocation in Federated Learningcited by 389
  5. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  6. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 347
  7. 2018Horovod: fast and easy distributed deep learning in TensorFlowcited by 275
  8. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  9. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 202
  10. 2019Adaptive Gradient Methods with Dynamic Bound of Learning Ratecited by 181
  11. 2017Revisiting Distributed Synchronous SGDcited by 142
  12. 2018A generic framework for privacy preserving deep learningcited by 103
  13. 2018Revisiting Small Batch Training for Deep Neural Networkscited by 100
  14. 2019Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregationcited by 99
  15. 2020Training Keyword Spotting Models on Non-IID Data with Federated Learningcited by 36
  16. 2017AdaComp : Adaptive Residual Gradient Compression for Data-Parallel Distributed Trainingcited by 33
  17. 2020Accelerating Federated Learning over Reliability-Agnostic Clients in Mobile Edge Computing Systemscited by 17
  18. 2020Characterizing Impacts of Heterogeneity in Federated Learning upon Large-Scale Smartphone Datacited by 9
  19. 2020Heterogeneous Data-Aware Federated Learningcited by 9

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 2.

4 papers.

  1. 2021Towards Fairness-Aware Federated Learningcited by 57
  2. 2022FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clientscited by 46
  3. 2021Resource-Efficient Federated Learningcited by 26
  4. 2022Towards Energy-Aware Federated Learning on Battery-Powered Clientscited by 11