mlsanityData to 7 Oct 2026

On the Effectiveness of Adaptation Strategies for VLM-Based Federated Learning in Remote Sensing

arXiv 2608.04791 ↗First posted 5 Aug 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) enables collaborative training of deep learning models across decentralized image archives without requiring data centralization. This paradigm is particularly relevant in remote sensing (RS), where legal regulations, privacy concerns, and bandwidth constraints restrict data sharing. However, the presence of training data heterogeneity across clients (known as non-IID data) can impede convergence and limit the generalization capability of the aggregated global model. To mitigate the adverse effects of training data heterogeneity, vision-language models (VLMs) can be leveraged in FL due to their transferable representations, which have demonstrated robustness under distribution shifts. However, their large parameter size may substantially increase communication overhead and local computational complexity in federated settings. Therefore, it is crucial to select an appropriate VLM adaptation strategy that balances the generalization ability with the communication and computational constraints. To address this issue, in this paper, we present the first comparative study of VLM adaptation strategies for FL in the context of RS image classification. We investigate full fine-tuning, encoder-specific fine-tuning, prompt learning, and low-rank adaptation (LoRA) tuning, and analyze them with respect to three criteria: 1) generalization capability under non-IID data, 2) communication overhead, and 3) local computational complexity. Experiments on BigEarthNet-S2, EuroSAT, RESISC45, and ImageNet reveal distinct trade-offs between task specialization, cross-domain generalization, and efficiency. Based on our findings, we derive a guideline for the selection of an appropriate VLM adaptation strategy in FL for RS image classification under different operational constraints. The code of this work is publicly available at https://git.tu-berlin.de/rsim/FL-RS-VLM.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 27 references that are in the map. First posted on arXiv 5 Aug 2026.

Builds on

22 of 27 references.

References · 22

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  6. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  7. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 791
  8. 2022Patching open-vocabulary models by interpolating weightscited by 187
  9. 2022Finetune like you pretrain: Improved finetuning of zero-shot vision modelscited by 149
  10. 2022PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Modelcited by 133
  11. 2023FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learningcited by 70
  12. 2024reBEN: Refined BigEarthNet Dataset for Remote Sensing Image Analysiscited by 48
10 more
  1. 2024Global and Local Prompts Cooperation via Optimal Transport for Federated Learningcited by 45
  2. 2022FedKL: Tackling Data Heterogeneity in Federated Reinforcement Learning by Penalizing KL Divergencecited by 29
  3. 2024Harmonizing Generalization and Personalization in Federated Prompt Learningcited by 26
  4. 2024Federated Learning from Vision-Language Foundation Models: Theoretical Analysis and Methodcited by 21
  5. 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 12
  6. 2025FedRSClip: Federated Learning for Remote Sensing Scene Classification Using Vision-Language Modelscited by 11
  7. 2023Federated Learning Across Decentralized and Unshared Archives for Remote Sensing Image Classificationcited by 10
  8. 2025FAA-CLIP: Federated Adversarial Adaptation of CLIPcited by 5
  9. 2025FedVLM: Scalable Personalized Vision-Language Models through Federated Learningcited by 2
  10. 2025CAPT: Class-Aware Prompt Tuning for Federated Long-Tailed Learning with Vision-Language Modelcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Decoupled Training with Local Reinforcement Fine-Tuning in Federated Learning
  2. 2025Vision-Language Modeling Meets Remote Sensing: Models, Datasets and Perspectives
  3. 2024FLoRA: Enhancing Vision-Language Models with Parameter-Efficient Federated Learning
  4. 2025TOFA: Training-Free One-Shot Federated Adaptation for Vision-Language Models
  5. 2025FUSE-RSVLM: Feature Fusion Vision-Language Model for Remote Sensing
  6. 2025FedVLMBench: Benchmarking Federated Fine-Tuning of Vision-Language Models
  7. 2023RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing
  8. 2024Advancements in Visual Language Models for Remote Sensing: Datasets, Capabilities, and Enhancement Techniques