mlsanityData to 7 Oct 2026

FusionRS: A Large-Scale RGB-Infrared-Style Remote Sensing Dataset for Cross-Modal Vision-Language Learning

arXiv 2606.17020 ↗First posted 15 Jun 2026cs.CV cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Remote sensing vision-language models have advanced Earth observation, but available large-scale vision-language resources remain RGB-centered, leaving complementary infrared information underexplored. Infrared observations provide distinctive intensity structures, object boundaries, and illumination-invariant cues that complement conventional RGB imagery, yet large-scale RGB-infrared-text resources remain scarce. We introduce FusionRS, the first large-scale RGB-infrared-style-text dataset for controlled dual-modal remote sensing vision-language learning. It contains 600,000 spatially aligned pairs created by translating diverse public RGB remote sensing images into infrared-style counterparts. Each pair retains a conventional scene caption, and a curated subset adds 45,913 IR-aware captions describing observable intensity, contrast, texture, and structure while preserving scene semantics. We train CLIP-style models for RGB-infrared-style-text alignment and adapt a generative vision-language model with mixed task-conditioned caption supervision. Evaluation covers cross-modal retrieval, scaling and supervision ablations, sensor-captured transfer, and strictly held-out captioning and VQA. FusionRS substantially improves RGB-infrared-style alignment and infrared-to-text retrieval over RGB-only and non-IR-aware settings. Ablations show that IR-aware captions improve task-conditioned infrared description, demonstrating the value of modality-specific supervision. FusionRS provides a scalable foundation for controlled RGB-infrared remote sensing vision-language learning.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 16 of its 30 references that are in the map. First posted on arXiv 15 Jun 2026.

Builds on

16 of 30 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025IRGPT: Understanding Real-world Infrared Image with Bi-cross-modal Curriculum on Large-scale Benchmark19 Jul
  2. 2023SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing20 Dec
  3. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensing24 Nov
  4. 2023RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing20 Jun
  5. 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensing19 Jun

Other references · 11

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  3. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  4. 2022Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detectioncited by 313
  5. 2020RSVQA: Visual Question Answering for Remote Sensing Datacited by 197
  6. 2017Exploring Models and Data for Remote Sensing Image Caption Generationcited by 195
  7. 2020Drone-based RGB-Infrared Cross-Modality Vehicle Detection via Uncertainty-Aware Learningcited by 150
  8. 2022HIT-UAV: A high-altitude infrared thermal dataset for Unmanned Aerial Vehicle-based object detectioncited by 31
  9. 2023Edge-guided Multi-domain RGB-to-TIR image Translation for Training Vision Tasks with Challenging Labelscited by 30
  10. 2024Caltech Aerial RGB-Thermal Dataset in the Wildcited by 19
  11. 2025DiffV2IR: Visible-to-Infrared Diffusion Model via Vision-Language Understandingcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026MonoIR-RS: Infrared Remote Sensing Vision-Language Learning with CLIP and VLM Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model
  2. 2024Mind the Modality Gap: Towards a Remote Sensing Vision-Language Model via Cross-modal Alignment
  3. 2026SkyNative: A Native Multimodal Architecture for Remote Sensing Vision-Language Understanding
  4. 2026Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks
  5. 2024RSTeller: Scaling Up Visual Language Modeling in Remote Sensing with Rich Linguistic Semantics from Openly Available Data and Large Language Models
  6. 2023RSGPT: A Remote Sensing Vision Language Model and Benchmark
  7. 2026Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?
  8. 2024VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding