mlsanityData to 14 Oct 2026

FedPrompt: Communication-Efficient and Privacy Preserving Prompt Tuning in Federated Learning

arXiv 2208.12268 ↗First posted 25 Aug 2022cs.LG cs.AI cs.CRcited by 91

On the map: LLM safety and security → Federated unlearning and poisoning

Abstract (arXiv)

Federated learning (FL) has enabled global model training on decentralized data in a privacy-preserving way by aggregating model updates. However, for many natural language processing (NLP) tasks that utilize pre-trained language models (PLMs) with large numbers of parameters, there are considerable communication costs associated with FL. Recently, prompt tuning, which tunes some soft prompts without modifying PLMs, has achieved excellent performance as a new learning paradigm. Therefore we want to combine the two methods and explore the effect of prompt tuning under FL. In this paper, we propose "FedPrompt" to study prompt tuning in a model split aggregation way using FL, and prove that split aggregation greatly reduces the communication cost, only 0.01% of the PLMs' parameters, with little decrease on accuracy both on IID and Non-IID data distribution. This improves the efficiency of FL method while also protecting the data privacy in prompt tuning. In addition, like PLMs, prompts are uploaded and downloaded between public platforms and personal users, so we try to figure out whether there is still a backdoor threat using only soft prompts in FL scenarios. We further conduct backdoor attacks by data poisoning on FedPrompt. Our experiments show that normal backdoor attack can not achieve a high attack success rate, proving the robustness of FedPrompt. We hope this work can promote the application of prompt in FL and raise the awareness of the possible security threats.

Key facts. Cited by 91 papers in the map (data through 14 Oct 2026). Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security. Builds on 32 of its 50 references that are in the map. First posted on arXiv 25 Aug 2022.

Builds on

32 of 50 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  10. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  11. 2016Deep Learning with Differential Privacycited by 2,728
  12. 2021Model-Contrastive Federated Learningcited by 766
20 more
  1. 2018How To Backdoor Federated Learningcited by 765
  2. 2021GPT Understands, Toocited by 716
  3. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
  4. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 689
  5. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  6. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 520
  7. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 459
  8. 2019Federated Machine Learning: Concept and Applicationscited by 412
  9. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 356
  10. 2017Adversarial Examples: Attacks and Defenses for Deep Learningcited by 351
  11. 2018Large Scale Crowdsourcing and Characterization of Twitter Abusive Behaviorcited by 270
  12. 2021PTR: Prompt Tuning with Rules for Text Classificationcited by 238
  13. 2021PPT: Pre-trained Prompt Tuning for Few-shot Learningcited by 232
  14. 2021WARP: Word-level Adversarial ReProgrammingcited by 221
  15. 2019SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)cited by 207
  16. 2019Federated Learning with Bayesian Differential Privacycited by 60
  17. 2021Adversarial Example Detection for DNN Models: A Review and Experimental Comparisoncited by 52
  18. 2021Scaling Federated Learning for Fine-tuning of Large Language Modelscited by 36
  19. 2021Benchmarking Differential Privacy and Federated Learning for BERT Modelscited by 28
  20. 2021Efficient-FedRec: Efficient Federated Learning Framework for Privacy-Preserving News Recommendationcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 12; 2024: 25; 2025: 37; 2026: 16.

91 papers.

Sort:

By area:

  1. 2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 84
  2. 2023Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimizationcited by 65
  3. 2024Advances and Open Challenges in Federated Foundation Modelscited by 36
  4. 2024The Future of Large Language Model Pre-training is Federatedcited by 30
  5. 2024A Survey on Efficient Federated Learning Methods for Foundation Model Trainingcited by 29
  6. 2024Federated Large Language Models: Current Progress and Future Directionscited by 27
  7. 2023FwdLLM: Efficient FedLLM using Forward Gradientcited by 26
  8. 2023FedBPT: Efficient Federated Black-box Prompt Tuning for Large Language Modelscited by 26
  9. 2023Grounding Foundation Models through Federated Transfer Learning: A General Frameworkcited by 24
  10. 2024Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Modelscited by 23
  11. 2024Federated Learning from Vision-Language Foundation Models: Theoretical Analysis and Methodcited by 22
  12. 2025A Survey on Privacy Risks and Protection in Large Language Modelscited by 21
38 more
  1. 2025Prompt-based Adaptation in Large-scale Vision Models: A Surveycited by 21
  2. 2025A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learningcited by 17
  3. 2025FlexOlmo: Open Language Models for Flexible Data Usecited by 15
  4. 2025Decentralized Low-Rank Fine-Tuning of Large Language Modelscited by 14
  5. 2024Federated Learning Priorities Under the European Union Artificial Intelligence Actcited by 13
  6. 2023Sweeping Heterogeneity with Smart MoPs: Mixture of Prompts for LLM Task Adaptationcited by 12
  7. 2024Open Challenges and Opportunities in Federated Foundation Models Towards Biomedical Healthcarecited by 12
  8. 2024Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Modelscited by 12
  9. 2022Exploring Parameter-Efficient Fine-Tuning to Enable Foundation Models in Federated Learningcited by 11
  10. 2023Tunable Soft Prompts are Messengers in Federated Learningcited by 10
  11. 2023Profit: Benchmarking Personalization and Robustness Trade-off in Federated Prompt Tuningcited by 9
  12. 2023ZooPFL: Exploring Black-box Foundation Models for Personalized Federated Learningcited by 9
  13. 2024DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptationcited by 9
  14. 2025Probabilistic Federated Prompt-Tuning with Non-IID and Imbalanced Datacited by 9
  15. 2025Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuningcited by 9
  16. 2026FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRAcited by 8
  17. 2025Fed-HeLLo: Efficient Federated Foundation Model Fine-Tuning with Heterogeneous LoRA Allocationcited by 7
  18. 2023Adapter-based Selective Knowledge Distillation for Federated Multi-domain Meeting Summarizationcited by 6
  19. 2024Federated Class-Incremental Learning with Hierarchical Generative Prototypescited by 6
  20. 2024Fisher Information-based Efficient Curriculum Federated Learning with Large Language Modelscited by 6
  21. 2025Adaptive LoRA Experts Allocation and Selection for Federated Fine-Tuningcited by 6
  22. 2023Leveraging Foundation Models to Improve Lightweight Clients in Federated Learningcited by 5
  23. 2024FedRDMA: Communication-Efficient Cross-Silo Federated LLM via Chunked RDMA Transmissioncited by 5
  24. 2024SFPrompt: Communication-Efficient Split Federated Fine-Tuning for Large Pre-Trained Models over Resource-Limited Devicescited by 5
  25. 2024FedPT: Federated Proxy-Tuning of Large Language Models on Resource-Constrained Edge Devicescited by 5
  26. 2024Federated Co-tuning Framework for Large and Small Language Modelscited by 5
  27. 2024Adaptive Parameter-Efficient Federated Fine-Tuning on Heterogeneous Devicescited by 5
  28. 2025Privacy-Preserving Personalized Federated Prompt Learning for Multimodal Large Language Modelscited by 5
  29. 2026Revisiting Backdoor Threat in Federated Instruction Tuning from a Signal Aggregation Perspectivecited by 5
  30. 2026Rethinking LoRA for Privacy-Preserving Federated Learning in Large Modelscited by 5
  31. 2024Save It All: Enabling Full Parameter Tuning for Federated Large Language Models via Cycle Block Gradient Descentcited by 4
  32. 2025Towards Harnessing the Collaborative Power of Large and Small Models for Domain Taskscited by 4
  33. 2025Patronus: Identifying and Mitigating Transferable Backdoors in Pre-trained Language Modelscited by 4
  34. 2024PPFedIT: Towards Privacy-Preserving Federated Instruction Tuning with Few-shot Local Examplescited by 3
  35. 2024FedSpaLLM: Federated Pruning of Large Language Modelscited by 3
  36. 2025A Survey on Foundation Models for Personalized Federated Intelligencecited by 3
  37. 2025FedRS-Bench: Realistic Federated Learning Datasets and Benchmarks in Remote Sensingcited by 3
  38. 2025Federated Domain Generalization with Domain-specific Soft Prompts Generationcited by 3

50 most cited of 91 All 91 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Privacy-preserving Prompt Personalization in Federated Learning for Multimodal Large Language Models
  2. 2024FedHPL: Efficient Heterogeneous Federated Learning with Prompt Tuning and Logit Distillation
  3. 2024Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security
  4. 2023Gradient-Free Privacy Leakage in Federated Language Models through Selective Weight Tampering
  5. 2025FedRand: Enhancing Privacy in Federated Learning with Randomized LoRA Subparameter Updates
  6. 2025Federated Fine-Tuning of LLMs: Framework Comparison and Research Directions
  7. 2024Analysis of Privacy Leakage in Federated Large Language Models
  8. 2025Can Federated Learning Safeguard Private Data in LLM Training? Vulnerabilities, Attacks, and Defense Evaluation