mlsanityData to 14 Oct 2026

Unlocking Multi-Spectral Data for Multi-Modal Models with Guided Inputs and Chain-of-Thought Reasoning

arXiv 2604.21032 ↗First posted 22 Apr 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Multi-spectral imagery is a valuable input signal for Remote Sensing applications, such as land-use and land-cover classification and environmental monitoring. However, generalist Large Multi-modal Models (LMMs) are typically trained on RGB images, limiting their applicability to the RGB domain. At the same time, training multi-spectral multi-modal models is expensive and produces uniquely specialized models. To address this, we propose a novel training-free approach that introduces multi-spectral data within the inference pipeline of standard RGB-only LMMs, allowing large gains in performance. Our approach leverages the LMMs' understanding of the visual space by adapting non-RGB inputs to that space and injecting domain-specific information and Chain-of-Thought reasoning as instructions. We demonstrate this with the Gemini 2.5 model and observe strong Zero-Shot performance gains on popular Remote Sensing benchmarks. These results highlight the potential for geospatial professionals to leverage powerful generalist models for specialized sensor inputs, benefiting from rich reasoning capabilities grounded in specialized data.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 38 references that are in the map. First posted on arXiv 22 Apr 2026.

Builds on

31 of 38 references.

References · 31

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  4. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  5. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 408
  6. 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensingcited by 363
  7. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensingcited by 328
  8. 2022Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learningcited by 308
  9. 2023SpectralGPT: Spectral Remote Sensing Foundation Modelcited by 252
  10. 2019BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understandingcited by 234
  11. 2023SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagerycited by 205
  12. 2021SpectralFormer: Rethinking Hyperspectral Image Classification with Transformerscited by 169
19 more
  1. 2023CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoderscited by 169
  2. 2022SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understandingcited by 167
  3. 2023Foundation Models for Generalist Geospatial Artificial Intelligencecited by 149
  4. 2025AlphaEarth Foundations: An embedding field model for accurate and efficient global mapping from sparse label datacited by 137
  5. 2023SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensingcited by 125
  6. 2023Towards Geospatial Foundation Models via Continual Pretrainingcited by 109
  7. 2020A Generalizable and Accessible Approach to Machine Learning with Global Satellite Imagerycited by 88
  8. 2024MMEarth: Exploring Multi-Modal Pretext Tasks For Geospatial Representation Learningcited by 77
  9. 2024HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Modelcited by 73
  10. 2023Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignmentcited by 71
  11. 2024OmniSat: Self-Supervised Modality Fusion for Earth Observationcited by 63
  12. 2023A Billion-scale Foundation Model for Remote Sensing Imagescited by 62
  13. 2024Vision Foundation Models in Remote Sensing: A Surveycited by 54
  14. 2024Good at captioning, bad at counting: Benchmarking GPT-4V on Earth observation datacited by 39
  15. 2024Label Propagation for Zero-shot Classification with Vision-Language Modelscited by 28
  16. 2023EarthPT: a time series foundation model for Earth Observationcited by 14
  17. 2025Beyond the Visible: Multispectral Vision-Language Learning for Earth Observationcited by 8
  18. 2025A Recipe for Improving Remote Sensing VLM Zero Shot Generalizationcited by 7
  19. 2025Zero-Shot Multi-Spectral Learning: Reimagining a Generalist Multimodal Gemini 2.5 Model for Remote Sensing Applicationscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond RGB: Benchmarking and Enhancing MLLMs for Hyperspectral Image Understanding via Training-Free Reasoning Framework
  2. 2026Spectrally Distilled Representations Aligned with Instruction-Augmented LLMs for Satellite Imagery
  3. 2024From Pixels to Prose: Advancing Multi-Modal Language Models for Remote Sensing
  4. 2026FusionRS: A Large-Scale RGB-Infrared-Style Remote Sensing Dataset for Cross-Modal Vision-Language Learning
  5. 2024RS-GPT4V: A Unified Multimodal Instruction-Following Dataset for Remote Sensing Image Understanding
  6. 2025Towards Faithful Reasoning in Remote Sensing: A Perceptually-Grounded GeoSpatial Chain-of-Thought for Vision-Language Models
  7. 2024SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understanding
  8. 2025Leveraging Large-Scale Pretrained Spatial-Spectral Priors for General Zero-Shot Pansharpening