mlsanityData to 14 Oct 2026

Learnings from Federated Learning in the Real world

arXiv 2202.03925 ↗First posted 8 Feb 2022cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) applied to real world data may suffer from several idiosyncrasies. One such idiosyncrasy is the data distribution across devices. Data across devices could be distributed such that there are some "heavy devices" with large amounts of data while there are many "light users" with only a handful of data points. There also exists heterogeneity of data across devices. In this study, we evaluate the impact of such idiosyncrasies on Natural Language Understanding (NLU) models trained using FL. We conduct experiments on data obtained from a large scale NLU system serving thousands of devices and show that simple non-uniform device selection based on the number of interactions at each round of FL training boosts the performance of the model. This benefit is further amplified in continual FL on consecutive time periods, where non-uniform sampling manages to swiftly catch up with FL methods using all data at once.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 13 of its 20 references that are in the map. First posted on arXiv 8 Feb 2022.

Builds on

13 of 20 references.

References · 13

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2020Adaptive Federated Optimizationcited by 1,046
  4. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  5. 2018Progress & Compress: A scalable framework for continual learningcited by 520
  6. 2019Compacting, Picking and Growing for Unforgetting Continual Learningcited by 207
  7. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 198
  8. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
  9. 2020Federated Continual Learning with Weighted Inter-client Transfercited by 161
  10. 2020Optimal Client Sampling for Federated Learningcited by 124
  11. 2020Towards Flexible Device Participation in Federated Learningcited by 65
  12. 2021Game of Gradients: Mitigating Irrelevant Clients in Federated Learningcited by 35
1 more
  1. 2020Dynamic Federated Learningcited by 12

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 1; 2025: 2.

6 papers.

  1. 2023Data-Aware Gradient Compression for FL in Communication-Constrained Mobile Computingcited by 5
  2. 2023Project Florida: Federated Learning Made Easycited by 2
  3. 2025Towards One-shot Federated Learning: Advances, Challenges, and Future Directionscited by 2
  4. 2023A Comprehensive View of Personalized Federated Learning on Heterogeneous Clinical Datasetscited by 1
  5. 2024Variational Bayes for Federated Continual Learningcited by 1
  6. 2025Taming Latency and Bandwidth: A Theoretical Framework and Adaptive Algorithm for Communication-Constrained Trainingcited by 1