Global Prompt Refinement with Non-Interfering Attention Masking for One-Shot Federated Learning
Not on the map. Why?
Abstract (arXiv)
Federated Prompt Learning (FPL) enables communication-efficient adaptation by tuning lightweight prompts on top of frozen pre-trained models. Existing FPL methods typically rely on global information, which is only available after the second training round, to facilitate collaboration among client models. Therefore, they are inherently dependent on multi-round communication to fully exhibit their strengths. Moreover, existing one-shot federated learning methods typically focus on fitting seen tasks, but lack cross-task generalization. To bridge this gap, we propose the Global Prompt Refinement with Non-Interfering Attention Masking (GPR-NIAM) method for one-shot FPL. The core idea is to design a masking mechanism that restricts excessive interaction between the original text embeddings and the learnable prompt embeddings. GPR-NIAM achieves this through the collaboration of two key modules. Firstly, the attention isolation module suppresses attention from the learnable prompt tokens to the original text tokens, and reweights the reverse attention which preserves generalization across tasks. Secondly, the cross-silo collaborative refinement module integrates decentralized visual knowledge into a unified base and calibrates the global prompt through multi-source cross-modal knowledge alignment, further mitigating the inconsistency caused by data heterogeneity. Extensive experiments conducted on ten benchmark datasets under two tasks show that GPR-NIAM outperforms eight state-of-the-art methods in both class-level and domain-level generalization.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 41 of its 80 references that are in the map. First posted on arXiv 21 Sep 2025.
Builds on
41 of 80 references.
References · 41
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2013Describing Textures in the Wildcited by 2,125
- 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
- 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
- 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
- 2022PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Modelcited by 134
- 2022Federated Class-Incremental Learningcited by 127
- 2021DENSE: Data-Free One-Shot Federated Learningcited by 100
- 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 99
- 2022FedPrompt: Communication-Efficient and Privacy Preserving Prompt Tuning in Federated Learningcited by 91
- 2023TCP:Textual-based Class-aware Prompt tuning for Visual-Language Modelcited by 76
29 more
- 2023Read-only Prompt Optimization for Vision-Language Few-shot Learningcited by 58
- 2023Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generationcited by 54
- 2024Global and Local Prompts Cooperation via Optimal Transport for Federated Learningcited by 46
- 2022Voice-Face Homogeneity Tells Deepfakecited by 44
- 2024Advances and Open Challenges in Federated Foundation Modelscited by 36
- 2023Cross-Silo Prototypical Calibration for Federated Learning with Non-IID Datacited by 35
- 2023Exploring One-shot Semi-supervised Federated Learning with A Pre-trained Diffusion Modelcited by 33
- 2024DiPrompT: Disentangled Prompt Tuning for Multiple Latent Domain Generalization in Federated Learningcited by 32
- 2022Federated Adaptive Prompt Tuning for Multi-Domain Collaborative Learningcited by 29
- 2024Harmonizing Generalization and Personalization in Federated Prompt Learningcited by 27
- 2024Federated Learning from Vision-Language Foundation Models: Theoretical Analysis and Methodcited by 22
- 2022FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregationcited by 21
- 2024Federated Model Heterogeneous Matryoshka Representation Learningcited by 21
- 2023Exploiting Multi-Label Correlation in Label Distribution Learningcited by 20
- 2023Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learningcited by 20
- 2025Ten Challenging Problems in Federated Foundation Modelscited by 20
- 2024FedDEO: Description-Enhanced One-Shot Federated Learning with Diffusion Modelscited by 17
- 2024Revisiting Ensembling in One-Shot Federated Learningcited by 13
- 2025Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learningcited by 12
- 2023pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharingcited by 9
- 2024One-Shot Sequential Federated Learning for Non-IID Data by Enhancing Local Model Diversitycited by 8
- 2025Privacy-Preserving Personalized Federated Prompt Learning for Multimodal Large Language Modelscited by 5
- 2025Label Distribution Learning with Biased Annotations by Learning Multi-Label Representationcited by 5
- 2025Feature Distillation is the Better Choice for Model-Heterogeneous Federated Learningcited by 5
- 2025Federated Deconfounding and Debiasing Learning for Out-of-Distribution Generalizationcited by 4
- 2024KNN Transformer with Pyramid Prompts for Few-Shot Learningcited by 3
- 2025Global Intervention and Distillation for Federated Out-of-Distribution Generalizationcited by 3
- 2025LLM-empowered Dynamic Prompt Routing for Vision-Language Models Tuning under Long-Tailed Distributionscited by 2
- 2025FedMHO: Heterogeneous One-Shot Federated Learning Towards Resource-Constrained Edge Devicescited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.