# BLEND: Balancing Personalization vs. Generalization in Federated Vision-Language Models

arXiv: https://arxiv.org/abs/2610.12907 · HTML: https://mlsanity.com/paper/2610.12907/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 30 of its 43 references that are in the map.
- First posted on arXiv 9 Oct 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification](https://mlsanity.com/paper/2207.09519/) (2022)
- [PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Model](https://mlsanity.com/paper/2208.11625/) (2022)
- [Federated Visual Classification with Real-World Data Distribution](https://mlsanity.com/paper/2003.08082/) (2020)
- [TCP:Textual-based Class-aware Prompt tuning for Visual-Language Model](https://mlsanity.com/paper/2311.18231/) (2023)
- [FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning](https://mlsanity.com/paper/2308.12305/) (2023)
- [Two Effects, One Trigger: On the Modality Gap, Object Bias, and Information Imbalance in Contrastive Vision-Language Models](https://mlsanity.com/paper/2404.07983/) (2024)
- [Global and Local Prompts Cooperation via Optimal Transport for Federated Learning](https://mlsanity.com/paper/2403.00041/) (2024)
- [MMRL: Multi-Modal Representation Learning for Vision-Language Models](https://mlsanity.com/paper/2503.08497/) (2025)
- [Learning Domain Invariant Prompt for Vision-Language Models](https://mlsanity.com/paper/2212.04196/) (2022)
- [Progressive Visual Prompt Learning with Contrastive Feature Re-formation](https://mlsanity.com/paper/2304.08386/) (2023)
- [Harmonizing Generalization and Personalization in Federated Prompt Learning](https://mlsanity.com/paper/2405.09771/) (2024)
- [Federated Learning from Vision-Language Foundation Models: Theoretical Analysis and Method](https://mlsanity.com/paper/2409.19610/) (2024)
- [DPC: Dual-Prompt Collaboration for Tuning Vision-Language Models](https://mlsanity.com/paper/2503.13443/) (2025)
- [Mixture of Experts Made Personalized: Federated Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2410.10114/) (2024)
- [Efficient Test-Time Prompt Tuning for Vision-Language Models](https://mlsanity.com/paper/2408.05775/) (2024)
- [Multi-Modal Multi-Task (M3T) Federated Foundation Models for Embodied AI: Potentials and Challenges for Edge Integration](https://mlsanity.com/paper/2505.11191/) (2025)
- [Bayesian Principles Improve Prompt Learning In Vision-Language Models](https://mlsanity.com/paper/2504.14123/) (2025)
- [pFedMMA: Personalized Federated Fine-Tuning with Multi-Modal Adapter for Vision-Language Models](https://mlsanity.com/paper/2507.05394/) (2025)
- [Prompt Tuning Vision Language Models with Margin Regularizer for Few-Shot Learning under Distribution Shifts](https://mlsanity.com/paper/2505.15506/) (2025)
- [Multi-Modal Multi-Task Federated Foundation Models for Next-Generation Extended Reality Systems: Towards Privacy-Preserving Distributed Intelligence in AR/VR/MR](https://mlsanity.com/paper/2506.05683/) (2025)
- [TriplePlay: Enhancing Federated Learning with CLIP for Non-IID Data and Resource Efficiency](https://mlsanity.com/paper/2409.05347/) (2024)
- [LLM-empowered Dynamic Prompt Routing for Vision-Language Models Tuning under Long-Tailed Distributions](https://mlsanity.com/paper/2508.15688/) (2025)
- [Hierarchical Federated Foundation Models over Wireless Networks for Multi-Modal Multi-Task Intelligence: Integration of Edge Learning with D2D/P2P-Enabled Fog Learning Architectures](https://mlsanity.com/paper/2509.03695/) (2025)
- [SemPT: Semantic Prompt Tuning for Vision-Language Models](https://mlsanity.com/paper/2508.10645/) (2025)

## Abstract (arXiv)

Federated vision-language models (FedVLMs) represent one of the emerging frontiers of machine learning, aiming to integrate federated learning within the fine-tuning pipelines of vision-language models (VLMs). In this work, we introduce BLEND, a new framework designed to jointly enhance the personalization and generalization capabilities of FedVLMs. BLEND proposes a selective VLM parameter fine-tuning and aggregation strategy that consists of (i) global vision and text adapters shared across clients, and (ii) a personalized vision projection adapter tailored to each client. BLEND further introduces a novel loss function that blends the outputs of the global and personalized adapters to achieve personalization, while simultaneously promoting generalization through a principled anchor design implemented via the projection heads of the zero-shot VLM. Extensive experiments demonstrate that BLEND consistently outperforms state-of-the-art baselines, highlighting its ability to effectively balance personalization and generalization across diverse system settings. (Code is available at: https://github.com/payamsiabd/BLEND.git)