mlsanityData to 7 Oct 2026

Federated Learning in Practice: Reflections and Projections

arXiv 2410.08892 ↗First posted 11 Oct 2024cs.LG cs.AI cs.CRcited by 15

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a machine learning technique that enables multiple entities to collaboratively learn a shared model without exchanging their local data. Over the past decade, FL systems have achieved substantial progress, scaling to millions of devices across various learning domains while offering meaningful differential privacy (DP) guarantees. Production systems from organizations like Google, Apple, and Meta demonstrate the real-world applicability of FL. However, key challenges remain, including verifying server-side DP guarantees and coordinating training across heterogeneous devices, limiting broader adoption. Additionally, emerging trends such as large (multi-modal) models and blurred lines between training, inference, and personalization challenge traditional FL frameworks. In response, we propose a redefined FL framework that prioritizes privacy principles rather than rigid definitions. We also chart a path forward by leveraging trusted execution environments and open-source ecosystems to address these challenges and facilitate future advancements in FL.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 55 of its 83 references that are in the map. First posted on arXiv 11 Oct 2024.

Builds on

55 of 83 references.

References · 55

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  8. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  9. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  10. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  11. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  12. 2019Advances and Open Problems in Federated Learningcited by 3,021
38 more
  1. 2016Deep Learning with Differential Privacycited by 2,682
  2. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  3. 2023PaLM 2 Technical Reportcited by 1,154
  4. 2020Adaptive Federated Optimizationcited by 1,036
  5. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  6. 2019Towards Federated Learning at Scale: System Designcited by 826
  7. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  8. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  9. 2019Federated Machine Learning: Concept and Applicationscited by 409
  10. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 382
  11. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
  12. 2021A Field Guide to Federated Optimizationcited by 312
  13. 2021Practical and Private (Deep) Learning without Sampling or Shufflingcited by 169
  14. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  15. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
  16. 2021When the Curious Abandon Honesty: Federated Learning Is Not Privatecited by 126
  17. 2019Federated Learning for Emoji Prediction in a Mobile Keyboardcited by 115
  18. 2024Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Modelscited by 114
  19. 2021Papaya: Practical, Private, and Scalable Federated Learningcited by 89
  20. 2020Improving Image Captioning with Better Use of Captionscited by 84
  21. 2021Federated Evaluation and Tuning for On-Device Personalization: System Design & Applicationscited by 84
  22. 2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 83
  23. 2021The Skellam Mechanism for Differentially Private Federated Learningcited by 66
  24. 2023Federated Learning of Gboard Language Models with Differential Privacycited by 64
  25. 2023Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Modelscited by 49
  26. 2023(Amplified) Banded Matrix Factorization: A unified approach to private trainingcited by 49
  27. 2023User Inference Attacks on Large Language Modelscited by 36
  28. 2024Privacy-Preserving Instructions for Aligning Large Language Modelscited by 34
  29. 2021Local Adaptivity in Federated Learning: Convergence and Consistencycited by 31
  30. 2024The Future of Large Language Model Pre-training is Federatedcited by 30
  31. 2024A Survey on Efficient Federated Learning Methods for Foundation Model Trainingcited by 29
  32. 2024Federated Large Language Models: Current Progress and Future Directionscited by 22
  33. 2024Prompt Public Large Language Models to Synthesize Data for Private On-device Applicationscited by 17
  34. 2022Production federated keyword spotting via distillation, filtering, and joint federated-centralized trainingcited by 9
  35. 2023Federated Analytics: A surveycited by 9
  36. 2023Profit: Benchmarking Personalization and Robustness Trade-off in Federated Prompt Tuningcited by 9
  37. 2024A Hassle-free Algorithm for Private Learning in Practice: Don't Use Tree Aggregation, Use BLTscited by 9
  38. 2022Applied Federated Learning: Architectural Design for Robust and Efficient Learning in Privacy Aware Settingscited by 7

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 12; 2026: 2.

15 papers.

Sort:

By area:

  1. 2025Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMscited by 15
  2. 2024Interplay between Federated Learning and Explainable Artificial Intelligence: a Scoping Reviewcited by 6
  3. 2025Aequa: Fair Model Rewards in Collaborative Learning via Slimmable Networkscited by 6
  4. 2025Research in Collaborative Learning Does Not Serve Cross-Silo Federated Learning in Practicecited by 3
  5. 2025Technical and Legal Aspects of Federated Learning in Bioinformatics: Applications, Challenges and Opportunitiescited by 2
  6. 2025Synthesizing and Adapting Error Correction Data for Mobile Large Language Model Applicationscited by 2
  7. 2025Federated Learning from Molecules to Processes: A Perspectivecited by 2
  8. 2025WikiDBGraph: A Data Management Benchmark Suite for Collaborative Learning over Database Siloscited by 0
  9. 2025Progressive Size-Adaptive Federated Learning: A Comprehensive Framework for Heterogeneous Multi-Modal Data Systemscited by 0
  10. 2025Beyond Trade-offs: A Unified Framework for Privacy, Robustness, and Communication Efficiency in Federated Learningcited by 0
  11. 2025On the Evolution of Federated Post-Training Large Language Models: A Model Accessibility Viewcited by 0
  12. 2025Securing Private Federated Learning in a Malicious Setting: A Scalable TEE-Based Approach with Client Auditingcited by 0
3 more
  1. 2025FedFusion: Federated Learning with Diversity- and Cluster-Aware Encoders for Robust Adaptation under Label Scarcitycited by 0
  2. 2026FedPBS: Proximal-Balanced Scaling Federated Learning Model for Robust Personalized Training for Non-IID Datacited by 0
  3. 2026Agentic Federated Learning: The Future of Distributed Training Orchestrationcited by 0