mlsanityData to 7 Oct 2026

PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Model

arXiv 2208.11625 ↗First posted 24 Aug 2022cs.LGcited by 133

Not on the map. Why?

Cited by 28 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Quick global aggregation of effective distributed parameters is crucial to federated learning (FL), which requires adequate bandwidth for parameters communication and sufficient user data for local training. Otherwise, FL may cost excessive training time for convergence and produce inaccurate models. In this paper, we propose a brand-new FL framework, PromptFL, that replaces the federated model training with the federated prompt training, i.e., let federated participants train prompts instead of a shared model, to simultaneously achieve the efficient global aggregation and local training on insufficient data by exploiting the power of foundation models (FM) in a distributed way. PromptFL ships an off-the-shelf FM, i.e., CLIP, to distributed clients who would cooperatively train shared soft prompts based on very few local data. Since PromptFL only needs to update the prompts instead of the whole model, both the local training and the global aggregation can be significantly accelerated. And FM trained over large scale data can provide strong adaptation capability to distributed users tasks with the trained soft prompts. We empirically analyze the PromptFL via extensive experiments, and show its superiority in terms of system feasibility, user privacy, and performance.

Key facts. Cited by 133 papers in the map (data through 7 Oct 2026). Builds on 41 of its 65 references that are in the map. First posted on arXiv 24 Aug 2022.

Builds on

41 of 65 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  9. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  10. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  11. 2019Advances and Open Problems in Federated Learningcited by 3,021
  12. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
29 more
  1. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  2. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  3. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  4. 2018Federated Learning with Non-IID Datacited by 1,099
  5. 2019Deep Leakage from Gradientscited by 1,004
  6. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  7. 2019Towards Federated Learning at Scale: System Designcited by 826
  8. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  9. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  10. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  11. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  12. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  13. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
  14. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  15. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  16. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
  17. 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
  18. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 377
  19. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
  20. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  21. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  22. 2021ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learningcited by 253
  23. 2021WARP: Word-level Adversarial ReProgrammingcited by 220
  24. 2019BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learningcited by 115
  25. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
  26. 2021CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIPcited by 88
  27. 2019Decentralized Federated Learning: A Segmented Gossip Approachcited by 63
  28. 2021Personalized Federated Learning with Contextualized Generalizationcited by 23
  29. 2022On the Difficulty of Defending Self-Supervised Learning against Model Extractioncited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 24; 2024: 33; 2025: 51; 2026: 22.

133 papers.

Sort:

By area:

  1. 2024Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Modelscited by 114
  2. 2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 83
  3. 2024Selective Aggregation for Low-Rank Adaptation in Federated Learningcited by 80
  4. 2023FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learningcited by 70
  5. 2023FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learningcited by 59
  6. 2023Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generationcited by 54
  7. 2024Global and Local Prompts Cooperation via Optimal Transport for Federated Learningcited by 45
  8. 2024Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Modelscited by 43
  9. 2023Prompt Federated Learning for Weather Forecasting: Toward Foundation Models on Meteorological Datacited by 40
  10. 2022FedTune: A Deep Dive into Efficient Federated Fine-Tuning with Pre-trained Transformerscited by 33
  11. 2023Exploring One-shot Semi-supervised Federated Learning with A Pre-trained Diffusion Modelcited by 33
  12. 2024DiPrompT: Disentangled Prompt Tuning for Multiple Latent Domain Generalization in Federated Learningcited by 32
38 more
  1. 2022Federated Adaptive Prompt Tuning for Multi-Domain Collaborative Learningcited by 29
  2. 2023Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured Learningcited by 29
  3. 2024PrE-Text: Training Language Models on Private Federated Data in the Age of LLMscited by 29
  4. 2023Integration of Large Language Models and Federated Learningcited by 28
  5. 2024Harmonizing Generalization and Personalization in Federated Prompt Learningcited by 26
  6. 2024Privacy Preserving Prompt Engineering: A Surveycited by 24
  7. 2023Federated Generative Learning with Foundation Modelscited by 23
  8. 2024Federated Large Language Models: Current Progress and Future Directionscited by 22
  9. 2024Federated Learning from Vision-Language Foundation Models: Theoretical Analysis and Methodcited by 21
  10. 2024PILoRA: Prototype Guided Incremental LoRA for Federated Class-Incremental Learningcited by 19
  11. 2023FedRA: A Random Allocation Strategy for Federated Tuning to Unleash the Power of Heterogeneous Clientscited by 18
  12. 2023Federated Prompt Learning for Weather Foundation Models on Devicescited by 17
  13. 2023Visual Prompt Based Personalized Federated Learningcited by 16
  14. 2023Text-driven Prompt Generation for Vision-Language Models in Federated Learningcited by 16
  15. 2024FLoRA: Enhancing Vision-Language Models with Parameter-Efficient Federated Learningcited by 16
  16. 2024Mixture of Experts Made Personalized: Federated Prompt Learning for Vision-Language Modelscited by 16
  17. 2024Personalized Adapter for Large Meteorology Model on Devices: Towards Weather Foundation Modelscited by 15
  18. 2025A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learningcited by 15
  19. 2024Vertical Federated Learning for Effectiveness, Security, Applicability: A Surveycited by 14
  20. 2024Leveraging Foundation Models for Multi-modal Federated Learning with Incomplete Modalitycited by 14
  21. 2024Synergizing Foundation Models and Federated Learning: A Surveycited by 14
  22. 2023Fed-CPrompt: Contrastive Prompt for Rehearsal-Free Federated Continual Learningcited by 12
  23. 2023Efficient Federated Prompt Tuning for Black-box Large Pre-trained Modelscited by 12
  24. 2024Open Challenges and Opportunities in Federated Foundation Models Towards Biomedical Healthcarecited by 12
  25. 2024CP-Prompt: Composition-Based Cross-modal Prompting for Domain-Incremental Continual Learningcited by 11
  26. 2024Accelerating Heterogeneous Federated Learning with Closed-form Classifierscited by 10
  27. 2023Profit: Benchmarking Personalization and Robustness Trade-off in Federated Prompt Tuningcited by 9
  28. 2023ZooPFL: Exploring Black-box Foundation Models for Personalized Federated Learningcited by 9
  29. 2025Vision Foundation Models in Medical Image Analysis: Advances and Challengescited by 9
  30. 2025Probabilistic Federated Prompt-Tuning with Non-IID and Imbalanced Datacited by 9
  31. 2025Latte: Collaborative Test-Time Adaptation of Vision-Language Models in Federated Learningcited by 9
  32. 2025Unlocking Vision-Language Models for Video Anomaly Detection via Fine-Grained Promptingcited by 8
  33. 2023The Role of Federated Learning in a Wireless World with Foundation Modelscited by 7
  34. 2024FACMIC: Federated Adaptative CLIP Model for Medical Image Classificationcited by 7
  35. 2025One-Shot Federated Learning with Classifier-Free Diffusion Modelscited by 7
  36. 2025Fed-HeLLo: Efficient Federated Foundation Model Fine-Tuning with Heterogeneous LoRA Allocationcited by 7
  37. 2025Capture Global Feature Statistics for One-Shot Federated Learningcited by 6
  38. 2023Leveraging Foundation Models to Improve Lightweight Clients in Federated Learningcited by 5

50 most cited of 133 All 133 (JSON)