mlsanityData to 7 Oct 2026

Lightweight Adaptation of General-Purpose VLMs for Multispectral and SAR Image Understanding

arXiv 2609.02187 ↗First posted 2 Sep 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

General-purpose vision-language models (VLMs) now support strong visual recognition, instruction following, and generation. However, most pretrained visual encoders are built around three-channel natural images and do not directly accommodate observations such as native multispectral measurements or synthetic aperture radar (SAR). Adapting VLMs to these sensors typically requires dedicated encoders and domain pretraining, slowing the reuse of stronger general-purpose checkpoints. We show that the multi-image interface of general-purpose VLMs offers a lightweight alternative. Our protocol renders each observation as five optical views and one SAR view, names them in the prompt, and adapts the language network and selected visual transformer blocks with LoRA. This exposes band composites, spectral indices, and radar backscatter through an existing visual interface. For land-cover recognition, structured supervision couples predicted classes with sensor evidence. We further construct preference pairs in which a true label is omitted while its supporting evidence is retained, encouraging complete predictions that remain consistent with the observations. On a balanced six-class land-cover benchmark derived from BigEarthNet-v2, the adapted Qwen3-VL reaches 0.8275 micro F1. The same input and adaptation protocol improves all four tested VLM architectures and transfers to Sen1Floods11 flood verification and BigEarthNet.txt captioning. Image removal and mismatch controls show that the adapted models use the supplied sensor observations. Together, these results demonstrate that VLMs can be repurposed for multispectral and SAR tasks through rendered inputs and compact LoRA adaptation, without training a new foundation model.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 27 references that are in the map. First posted on arXiv 2 Sep 2026.

Builds on

18 of 27 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks9 Jun
  2. 2025EarthMind: Leveraging Cross-Sensor Data for Advanced Earth Observation Interpretation with a Unified Multimodal LLM2 Jun
  3. 2024EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domain30 Jan
  4. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensing24 Nov
  5. 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensing19 Jun

Other references · 13

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  3. 2022Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learningcited by 302
  4. 2023SpectralGPT: Spectral Remote Sensing Foundation Modelcited by 250
  5. 2019BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understandingcited by 231
  6. 2025SmolVLM: Redefining small and efficient multimodal modelscited by 226
  7. 2021Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Datacited by 214
  8. 2023SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagerycited by 202
  9. 2023CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoderscited by 167
  10. 2022SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understandingcited by 164
  11. 2024Prithvi-EO-2.0: A Versatile Multi-Temporal Foundation Model for Earth Observation Applicationscited by 139
  12. 2022SSL4EO-S12: A Large-Scale Multi-Modal, Multi-Temporal Dataset for Self-Supervised Learning in Earth Observationcited by 82
1 more
  1. 2025TerraFM: A Scalable Foundation Model for Unified Multisensor Earth Observationcited by 25

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025SARVLM: A Vision Language Foundation Model for Semantic Understanding in SAR Imagery
  2. 2025SARLANG-1M: A Benchmark for Vision-Language Modeling in SAR Image Understanding
  3. 2026FUSAR-GPT : A Spatiotemporal Feature-Embedded and Two-Stage Decoupled Visual Language Model for SAR Imagery
  4. 2025Efficient Few-Shot Learning in Remote Sensing: Fusing Vision and Vision-Language Models
  5. 2025Vision-Language Modeling Meets Remote Sensing: Models, Datasets and Perspectives
  6. 2025SARChat-Bench-2M: A Multi-Task Vision-Language Benchmark for SAR Image Interpretation
  7. 2025SAR-TEXT: A Large-Scale SAR Image-Text Dataset Built with SAR-Narrator and A Progressive Learning Strategy for Downstream Tasks
  8. 2024From Pixels to Prose: Advancing Multi-Modal Language Models for Remote Sensing