mlsanityData to 14 Oct 2026

Federated Cross-Modal Style-Aware Prompt Generation

arXiv 2508.12399 ↗First posted 17 Aug 2025cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Prompt learning has propelled vision-language models like CLIP to excel in diverse tasks, making them ideal for federated learning due to computational efficiency. However, conventional approaches that rely solely on final-layer features miss out on rich multi-scale visual cues and domain-specific style variations in decentralized client data. To bridge this gap, we introduce FedCSAP (Federated Cross-Modal Style-Aware Prompt Generation). Our framework harnesses low, mid, and high-level features from CLIP's vision encoder alongside client-specific style indicators derived from batch-level statistics. By merging intricate visual details with textual context, FedCSAP produces robust, context-aware prompt tokens that are both distinct and non-redundant, thereby boosting generalization across seen and unseen classes. Operating within a federated learning paradigm, our approach ensures data privacy through local training and global aggregation, adeptly handling non-IID class distributions and diverse domain-specific styles. Comprehensive experiments on multiple image classification datasets confirm that FedCSAP outperforms existing federated prompt learning methods in both accuracy and overall generalization.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 41 references that are in the map. First posted on arXiv 17 Aug 2025.

Builds on

28 of 41 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Harmonizing Generalization and Personalization in Federated Prompt Learning16 May
  2. 2023Improving Zero-Shot Generalization for CLIP with Synthesized Prompts14 Jul
  3. 2023Visual-Language Prompt Tuning with Knowledge-guided Context Optimization23 Mar
  4. 2022SuS-X: Training-Free Name-Only Transfer of Vision-Language Models28 Nov
  5. 2022Prompt-aligned Gradient for Prompt Tuning30 May
  6. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  7. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 21

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2017Squeeze-and-Excitation Networkscited by 4,789
  4. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
  5. 2022MaPLe: Multi-modal Prompt Learningcited by 687
  6. 2013Network In Networkcited by 605
  7. 2021True Few-Shot Learning with Language Modelscited by 344
  8. 2022PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Modelcited by 134
  9. 2022FedPrompt: Communication-Efficient and Privacy Preserving Prompt Tuning in Federated Learningcited by 91
  10. 2023FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learningcited by 70
  11. 2022LASP: Text-to-Text Optimization for Language-Aware Soft Prompting of Vision & Language Modelscited by 59
  12. 2023Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generationcited by 54
9 more
  1. 2024Global and Local Prompts Cooperation via Optimal Transport for Federated Learningcited by 46
  2. 2022FedTP: Federated Learning by Transformer Personalizationcited by 36
  3. 2022Federated Adaptive Prompt Tuning for Multi-Domain Collaborative Learningcited by 29
  4. 2023APPLeNet: Visual Attention Parameterized Prompt Learning for Few-Shot Remote Sensing Image Generalization using CLIPcited by 24
  5. 2023Federated Prompt Learning for Weather Foundation Models on Devicescited by 17
  6. 2022Knowledge-Aware Federated Active Learning with Non-IID Datacited by 16
  7. 2021FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximationcited by 10
  8. 2023Fed-CO2: Cooperation of Online and Offline Models for Severe Data Heterogeneity in Federated Learningcited by 7
  9. 2023Cooperative Hardware-Prompt Learning for Snapshot Compressive Imagingcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025FedDEAP: Adaptive Dual-Prompt Tuning for Multi-Domain Federated Learning
  2. 2023Text-driven Prompt Generation for Vision-Language Models in Federated Learning
  3. 2025FedMVP: Federated Multimodal Visual Prompt Tuning for Vision-Language Models
  4. 2025Replication Study: Federated Text-Driven Prompt Generation for Vision-Language Models
  5. 2024Mixture of Experts Made Personalized: Federated Prompt Learning for Vision-Language Models
  6. 2025FLIP: Towards Comprehensive and Reliable Evaluation of Federated Prompt Learning
  7. 2023Learning to Prompt Your Domain for Vision-Language Models
  8. 2025FedMGP: Personalized Federated Learning with Multi-Group Text-Visual Prompts