mlsanityData to 7 Oct 2026

Papaya: Practical, Private, and Scalable Federated Learning

arXiv 2111.04877 ↗First posted 8 Nov 2021cs.LG cs.DCcited by 89

Not on the map. Why?

Abstract (arXiv)

Cross-device Federated Learning (FL) is a distributed learning paradigm with several challenges that differentiate it from traditional distributed learning, variability in the system characteristics on each device, and millions of clients coordinating with a central server being primary ones. Most FL systems described in the literature are synchronous - they perform a synchronized aggregation of model updates from individual clients. Scaling synchronous FL is challenging since increasing the number of clients training in parallel leads to diminishing returns in training speed, analogous to large-batch training. Moreover, stragglers hinder synchronous FL training. In this work, we outline a production asynchronous FL system design. Our work tackles the aforementioned issues, sketches of some of the system design challenges and their solutions, and touches upon principles that emerged from building a production FL system for millions of clients. Empirically, we demonstrate that asynchronous FL converges faster than synchronous FL when training across nearly one hundred million devices. In particular, in high concurrency settings, asynchronous FL is 5x faster and has nearly 8x less communication overhead than synchronous FL.

Key facts. Cited by 89 papers in the map (data through 7 Oct 2026). Builds on 16 of its 35 references that are in the map. First posted on arXiv 8 Nov 2021.

Builds on

16 of 35 references.

References · 16

  1. 2019Advances and Open Problems in Federated Learningcited by 3,021
  2. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  3. 2020Adaptive Federated Optimizationcited by 1,036
  4. 2019Towards Federated Learning at Scale: System Designcited by 826
  5. 2018LEAF: A Benchmark for Federated Settingscited by 780
  6. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  7. 2019Asynchronous Federated Optimizationcited by 264
  8. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 188
  9. 2016Practical Secure Aggregation for Federated Learning on User-Held Datacited by 178
  10. 2015Character-Aware Neural Language Modelscited by 129
  11. 2021Asynchronous Federated Learning on Heterogeneous Devices: A Surveycited by 93
  12. 2021Federated Evaluation and Tuning for On-Device Personalization: System Design & Applicationscited by 84
4 more
  1. 2020Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learningcited by 78
  2. 2021On Large-Cohort Training for Federated Learningcited by 73
  3. 2020IBM Federated Learning: an Enterprise Framework White Paper V0.1cited by 57
  4. 2021Secure Aggregation for Buffered Asynchronous Federated Learningcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 18; 2023: 25; 2024: 32; 2025: 7; 2026: 7.

89 papers.

Sort:

By area:

  1. 2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 83
  2. 2022FederatedScope: A Flexible Federated Learning Platform for Heterogeneitycited by 53
  3. 2023(Amplified) Banded Matrix Factorization: A unified approach to private trainingcited by 49
  4. 2023Federated Learning for Computationally-Constrained Heterogeneous Devices: A Surveycited by 40
  5. 2023On the Convergence of Federated Averaging with Cyclic Client Participationcited by 31
  6. 2022AFLGuard: Byzantine-robust Asynchronous Federated Learningcited by 29
  7. 2023TimelyFL: Heterogeneity-aware Asynchronous Federated Learning with Adaptive Partial Trainingcited by 27
  8. 2022CANIFE: Crafting Canaries for Empirical Privacy Measurement in Federated Learningcited by 24
  9. 2023FS-Real: Towards Real-World Cross-Device Federated Learningcited by 22
  10. 2022FedShuffle: Recipes for Better Use of Local Work in Federated Learningcited by 19
  11. 2022Towards Fair Federated Recommendation Learning: Characterizing the Inter-Dependence of System and Data Heterogeneitycited by 17
  12. 2023FLINT: A Platform for Federated Learning Integrationcited by 17
38 more
  1. 2023FedVS: Straggler-Resilient and Privacy-Preserving Vertical Federated Learning for Split Modelscited by 17
  2. 2024Prompt Public Large Language Models to Synthesize Data for Private On-device Applicationscited by 17
  3. 2023AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devicescited by 15
  4. 2024Federated Learning in Practice: Reflections and Projectionscited by 15
  5. 2022Optimizing the Communication-Accuracy Trade-off in Federated Learning with Rate-Distortion Theorycited by 14
  6. 2024The Federation Strikes Back: A Survey of Federated Learning Privacy Attacks, Defenses, Applications, and Policy Landscapecited by 13
  7. 2022Efficient and Light-Weight Federated Learning via Asynchronous Distributed Dropoutcited by 12
  8. 2024Open Challenges and Opportunities in Federated Foundation Models Towards Biomedical Healthcarecited by 12
  9. 2022A Survey on Participant Selection for Federated Learning in Mobile Networkscited by 11
  10. 2022Federated Select: A Primitive for Communication- and Memory-Efficient Federated Learningcited by 11
  11. 2023Async-HFL: Efficient and Robust Asynchronous Federated Learning in Hierarchical IoT Networkscited by 11
  12. 2024Recurrent Early Exits for Federated Learning with Heterogeneous Clientscited by 11
  13. 2023Asynchronous Federated Learning with Bidirectional Quantized Communications and Buffered Aggregationcited by 10
  14. 2023Federated Learning Across Decentralized and Unshared Archives for Remote Sensing Image Classificationcited by 10
  15. 2024DrJAX: Scalable and Differentiable MapReduce Primitives in JAXcited by 9
  16. 2024Noiseless Privacy-Preserving Decentralized Learningcited by 9
  17. 2024A Hassle-free Algorithm for Private Learning in Practice: Don't Use Tree Aggregation, Use BLTscited by 9
  18. 2022Pisces: Efficient Federated Learning via Guided Asynchronous Trainingcited by 8
  19. 2022Auxo: Efficient Federated Learning via Scalable Client Clusteringcited by 8
  20. 2023Green Federated Learningcited by 8
  21. 2024Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysiscited by 8
  22. 2022Applied Federated Learning: Architectural Design for Robust and Efficient Learning in Privacy Aware Settingscited by 7
  23. 2023Flame: Simplifying Topology Extension in Federated Learningcited by 7
  24. 2024COALA: A Practical and Vision-Centric Federated Learning Platformcited by 7
  25. 2022FedHPO-B: A Benchmark Suite for Federated Hyperparameter Optimizationcited by 6
  26. 2023Samplable Anonymous Aggregation for Private Federated Data Analysiscited by 6
  27. 2023FLAIM: AIM-based Synthetic Data Generation in the Federated Settingcited by 6
  28. 2024Boosting Asynchronous Decentralized Learning with Model Fragmentationcited by 6
  29. 2025Aequa: Fair Model Rewards in Collaborative Learning via Slimmable Networkscited by 6
  30. 2023FilFL: Client Filtering for Optimized Client Participation in Federated Learningcited by 5
  31. 2024Worldwide Federated Training of Language Modelscited by 5
  32. 2024FedStaleWeight: Buffered Asynchronous Federated Learning with Fair Aggregation via Staleness Reweightingcited by 5
  33. 2022FEL: High Capacity Learning for Recommendation and Ranking via Federated Ensemble Learningcited by 4
  34. 2022Reconciling Security and Communication Efficiency in Federated Learningcited by 4
  35. 2023FedML Parrot: A Scalable Federated Learning System via Heterogeneity-aware Scheduling on Sequential and Hierarchical Trainingcited by 4
  36. 2024Confidential Federated Computationscited by 4
  37. 2026Decoupled DiLoCo for Resilient Distributed Pre-trainingcited by 4
  38. 2022Towards cost-effective and resource-aware aggregation at Edge for Federated Learningcited by 3

50 most cited of 89 All 89 (JSON)