mlsanityData to 14 Oct 2026

Federated Learning Meets Natural Language Processing: A Survey

arXiv 2107.12603 ↗First posted 27 Jul 2021cs.CL cs.AI cs.DCcited by 43

Not on the map. Why?

Abstract (arXiv)

Federated Learning aims to learn machine learning models from multiple decentralized edge devices (e.g. mobiles) or servers without sacrificing local data privacy. Recent Natural Language Processing techniques rely on deep learning and large pre-trained language models. However, both big deep neural and language models are trained with huge amounts of data which often lies on the server side. Since text data is widely originated from end users, in this work, we look into recent NLP models and techniques which use federated learning as the learning framework. Our survey discusses major challenges in federated natural language processing, including the algorithm challenges, system challenges as well as the privacy issues. We also provide a critical review of the existing Federated NLP evaluation methods and tools. Finally, we highlight the current research gaps and future directions.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). Builds on 41 of its 72 references that are in the map. First posted on arXiv 27 Jul 2021.

Builds on

41 of 72 references.

References · 41

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2018Federated Learning with Non-IID Datacited by 1,108
  6. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  7. 2018Federated Learning for Mobile Keyboard Predictioncited by 572
  8. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 536
  9. 2015LSTM: A Search Space Odysseycited by 316
  10. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
  11. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 305
  12. 2018Dataset Distillationcited by 230
29 more
  1. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 204
  2. 2019SecureBoost: A Lossless Federated Learning Frameworkcited by 176
  3. 2019One-Shot Federated Learningcited by 125
  4. 2019Federated Learning for Emoji Prediction in a Mobile Keyboardcited by 118
  5. 2017Hello Edge: Keyword Spotting on Microcontrollerscited by 110
  6. 2020Distilled One-Shot Federated Learningcited by 109
  7. 2019Soft-Label Dataset Distillation and Text Dataset Distillationcited by 107
  8. 2018Federated Learning for Keyword Spottingcited by 95
  9. 2019Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation Systemcited by 79
  10. 2019Patient Clustering Improves Efficiency of Federated Machine Learning to predict mortality and hospital stay time using distributed Electronic Medical Recordscited by 72
  11. 2020FedHome: Cloud-Edge based Personalized Federated Learning for In-Home Health Monitoringcited by 61
  12. 2020Decentralizing Feature Extraction with Quantum Convolutional Neural Network for Automatic Speech Recognitioncited by 57
  13. 2019Federated Learning Of Out-Of-Vocabulary Wordscited by 51
  14. 2018Learning Private Neural Language Modeling with Attentive Aggregationcited by 49
  15. 2020Training Speech Recognition Models with Federated Learning: A Quality/Cost Frameworkcited by 46
  16. 2020Understanding Unintended Memorization in Federated Learningcited by 36
  17. 2021Scaling Federated Learning for Fine-tuning of Large Language Modelscited by 36
  18. 2019Federated Learning of N-gram Language Modelscited by 33
  19. 2020Communication-Efficient Federated Distillationcited by 28
  20. 2020FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learningcited by 24
  21. 2020Pretraining Federated Text Models for Next Word Predictioncited by 22
  22. 2017Federated Tensor Factorization for Computational Phenotypingcited by 21
  23. 2019Two-stage Federated Phenotyping and Patient Representation Learningcited by 19
  24. 2020Federated pretraining and fine tuning of BERT using clinical notes from multiple siloscited by 18
  25. 2021Robust Federated Learning with Attack-Adaptive Aggregationcited by 17
  26. 2017Factorization tricks for LSTM networkscited by 13
  27. 2021Scaling Neuroscience Research using Federated Learningcited by 10
  28. 2019Preserving Patient Privacy while Training a Predictive Model of In-hospital Mortalitycited by 7
  29. 2019Approximating probabilistic models as weighted finite automatacited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 11; 2023: 12; 2024: 11; 2025: 6; 2026: 2.

43 papers.

Sort:

By area:

  1. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 185
  2. 2024Advances and Open Challenges in Federated Foundation Modelscited by 36
  3. 2022Pretrained Models for Multilingual Federated Learningcited by 30
  4. 2023Towards Quantum Federated Learningcited by 20
  5. 2021Privacy enabled Financial Text Classification using Differential Privacy and Federated Learningcited by 18
  6. 2022Federated Optimization Algorithms with Random Reshuffling and Gradient Compressioncited by 18
  7. 2022Faster federated optimization under second-order similaritycited by 18
  8. 2023Beyond Exponentially Fast Mixing in Average-Reward Reinforcement Learning via Multi-Level Monte Carlo Actor-Criticcited by 18
  9. 2023Mutual Enhancement of Large and Small Language Models with Cross-Silo Knowledge Transfercited by 13
  10. 2022Federated Learning and Meta Learning: Approaches, Applications, and Directionscited by 12
  11. 2024Open Challenges and Opportunities in Federated Foundation Models Towards Biomedical Healthcarecited by 12
  12. 2022Federated Multilingual Models for Medical Transcript Analysiscited by 11
31 more
  1. 2023Stochastic Distributed Optimization under Average Second-order Similarity: Algorithms and Analysiscited by 11
  2. 2022A Review of Federated Learning in Energy Systemscited by 9
  3. 2023A Survey on Vulnerability of Federated Learning: A Learning Algorithm Perspectivecited by 9
  4. 2024Blockchain for Large Language Model Security and Safety: A Holistic Surveycited by 8
  5. 2023Hazards from Increasingly Accessible Fine-Tuning of Downloadable Foundation Modelscited by 7
  6. 2023Adapter-based Selective Knowledge Distillation for Federated Multi-domain Meeting Summarizationcited by 6
  7. 2025Multiplayer Federated Learning: Reaching Equilibrium with Less Communicationcited by 5
  8. 2024Open-Vocabulary Federated Learning with Multimodal Prototypingcited by 4
  9. 2022FewFedWeight: Few-shot Federated Learning Framework across Multiple NLP Taskscited by 3
  10. 2023Edge-assisted U-Shaped Split Federated Learning with Privacy-preserving for Internet of Thingscited by 3
  11. 2024MedLeak: Multimodal Medical Data Leakage in Secure Federated Learning with Crafted Modelscited by 3
  12. 2023Now It Sounds Like You: Learning Personalized Vocabulary On Devicecited by 2
  13. 2024Only Send What You Need: Learning to Communicate Efficiently in Federated Multilingual Machine Translationcited by 2
  14. 2024Collaborative Visual Place Recognition through Federated Learningcited by 2
  15. 2025Federated Learning-Enabled Hybrid Language Models for Communication-Efficient Token Transmissioncited by 2
  16. 2025Efficient Federated Learning with Timely Update Disseminationcited by 2
  17. 2022Federated Named Entity Recognitioncited by 1
  18. 2022FLVoogd: Robust And Privacy Preserving Federated Learningcited by 1
  19. 2022SPARTAN: Sparse Hierarchical Memory for Parameter-Efficient Transformerscited by 1
  20. 2023An In-Depth Evaluation of Federated Learning on Biomedical Natural Language Processingcited by 1
  21. 2024FedNE: Surrogate-Assisted Federated Neighbor Embedding for Dimensionality Reductioncited by 1
  22. 2025Adaptive Federated Distillation for Multi-Domain Non-IID Textual Datacited by 1
  23. 2022Federated Adversarial Training with Transformerscited by 0
  24. 2023Equitable-FL: Federated Learning with Sparsity for Resource-Constrained Environmentcited by 0
  25. 2024Unsupervised Federated Optimization at the Edge: D2D-Enabled Learning without Labelscited by 0
  26. 2024Privacy-preserving federated prediction of pain intensity change based on multi-center survey datacited by 0
  27. 2024Which PPML Would a User Choose? A Structured Decision Support Framework for Developers to Rank PPML Techniques Based on User Acceptance Criteriacited by 0
  28. 2025SEAFL: Enhancing Efficiency in Semi-Asynchronous Federated Learning through Adaptive Aggregation and Selective Trainingcited by 0
  29. 2025Assortment of Attention Heads: Accelerating Federated PEFT with Head Pruning and Strategic Client Selectioncited by 0
  30. 2026AdaBFL: Multi-Layer Defensive Adaptive Aggregation for Bzantine-Robust Federated Learningcited by 0
  31. 2026Theoretical Foundations of Communication-Efficient, Robust, and Practical Distributed and Federated Optimizationcited by 0