# DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Response

arXiv: https://arxiv.org/abs/2505.21089 · HTML: https://mlsanity.com/paper/2505.21089/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 43 of its 56 references that are in the map.
- First posted on arXiv 27 May 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 30 of 43)

- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [LISA: Reasoning Segmentation via Large Language Model](https://mlsanity.com/paper/2308.00692/) (2023)
- [LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models](https://mlsanity.com/paper/2407.07895/) (2024)
- [Ferret: Refer and Ground Anything Anywhere at Any Granularity](https://mlsanity.com/paper/2310.07704/) (2023)
- [VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks](https://mlsanity.com/paper/2305.11175/) (2023)
- [GLaMM: Pixel Grounding Large Multimodal Model](https://mlsanity.com/paper/2311.03356/) (2023)
- [Kimi-VL Technical Report](https://mlsanity.com/paper/2504.07491/) (2025)
- [AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classification](https://mlsanity.com/paper/1608.05167/) (2016)
- [GeoChat: Grounded Large Vision-Language Model for Remote Sensing](https://mlsanity.com/paper/2311.15826/) (2023)
- [SpectralGPT: Spectral Remote Sensing Foundation Model](https://mlsanity.com/paper/2311.07113/) (2023)
- [PixelLM: Pixel Reasoning with Large Multimodal Model](https://mlsanity.com/paper/2312.02228/) (2023)
- [RSGPT: A Remote Sensing Vision Language Model and Benchmark](https://mlsanity.com/paper/2307.15266/) (2023)
- [RSVQA: Visual Question Answering for Remote Sensing Data](https://mlsanity.com/paper/2003.07333/) (2020)
- [Exploring Models and Data for Remote Sensing Image Caption Generation](https://mlsanity.com/paper/1712.07835/) (2017)
- [FloodNet: A High Resolution Aerial Imagery Dataset for Post Flood Scene Understanding](https://mlsanity.com/paper/2012.02951/) (2020)
- [Multi-modal Learning with Missing Modality via Shared-Specific Feature Modelling](https://mlsanity.com/paper/2307.14126/) (2023)
- [xBD: A Dataset for Assessing Building Damage from Satellite Imagery](https://mlsanity.com/paper/1911.09296/) (2019)
- [RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Data](https://mlsanity.com/paper/2210.12634/) (2022)
- [VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding](https://mlsanity.com/paper/2406.12384/) (2024)
- [PSALM: Pixelwise SegmentAtion with Large Multi-Modal Model](https://mlsanity.com/paper/2403.14598/) (2024)

## Built on by (most cited first, 30 of 36)

- [SegEarth-OV3: Exploring SAM 3 for Open-Vocabulary Semantic Segmentation in Remote Sensing Images](https://mlsanity.com/paper/2512.08730/) (2025)
- [SegEarth-R2: Towards Comprehensive Language-guided Segmentation for Remote Sensing Images](https://mlsanity.com/paper/2512.20013/) (2025)
- [Towards Generative Location Awareness for Disaster Response: A Probabilistic Cross-view Geolocalization Approach](https://mlsanity.com/paper/2512.20056/) (2025)
- [TerraScope: Pixel-Grounded Visual Reasoning for Earth Observation](https://mlsanity.com/paper/2603.19039/) (2026)
- [Can LLM Agents Respond to Disasters? Benchmarking Heterogeneous Geospatial Reasoning in Emergency Operations](https://mlsanity.com/paper/2605.11633/) (2026)
- [EarthVL: A Progressive Earth Vision-Language Understanding and Generation Framework](https://mlsanity.com/paper/2601.02783/) (2026)
- [EarthSpatialBench: Benchmarking Spatial Reasoning Capabilities of Multimodal LLMs on Earth Imagery](https://mlsanity.com/paper/2602.15918/) (2026)
- [Experience-Driven Multi-Agent Systems Are Training-free Context-aware Earth Observers](https://mlsanity.com/paper/2602.02559/) (2026)
- [Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks](https://mlsanity.com/paper/2606.10819/) (2026)
- [CyPortQA: Benchmarking Multimodal Large Language Models for Cyclone Preparedness in Port Operation](https://mlsanity.com/paper/2508.15846/) (2025)
- [OmniEarth: A Benchmark for Evaluating Vision-Language Models in Geospatial Tasks](https://mlsanity.com/paper/2603.09471/) (2026)
- [Recov-Vision: Linking Street View Imagery and Vision-Language Models for Post-Disaster Recovery](https://mlsanity.com/paper/2509.20628/) (2025)
- [DisasterInsight: A Building-Centric Benchmark for Evaluating Vision--Language Models in Disaster Response](https://mlsanity.com/paper/2601.18493/) (2026)
- [RS-WorldModel: a Unified Model for Remote Sensing Understanding and Future Sense Forecasting](https://mlsanity.com/paper/2603.14941/) (2026)
- [NeSy-Route: A Neuro-Symbolic Benchmark for Constrained Route Planning in Remote Sensing](https://mlsanity.com/paper/2603.16307/) (2026)
- [CoRE-UIR: Prior-guided common and residual experts for efficient all-in-one remote sensing image restoration](https://mlsanity.com/paper/2607.27898/) (2026)
- [The Potential of Copernicus Satellites for Disaster Response: Retrieving Building Damage from Sentinel-1 and Sentinel-2](https://mlsanity.com/paper/2511.05461/) (2025)
- [Geo3DVQA: Evaluating Vision-Language Models for 3D Geospatial Reasoning from Aerial Imagery](https://mlsanity.com/paper/2512.07276/) (2025)
- [From Pixels to Semantics: A Multi-Stage AI Framework for Structural Damage Detection in Satellite Imagery](https://mlsanity.com/paper/2603.22768/) (2026)
- [RAPID: A Reproducible Multi-Agent Pipeline for Interpretable Disaster Damage Assessment from Satellite and Street-View Imagery](https://mlsanity.com/paper/2606.21819/) (2026)
- [VLCE: A Knowledge-Enhanced Framework for Image Description in Disaster Assessment](https://mlsanity.com/paper/2509.21609/) (2025)
- [ChangeQuery: Advancing Remote Sensing Change Analysis for Natural and Human-Induced Disasters from Visual Detection to Semantic Understanding](https://mlsanity.com/paper/2604.22333/) (2026)
- [Can Multimodal Large Language Models Truly Understand Small Objects?](https://mlsanity.com/paper/2604.22884/) (2026)
- [Survey on Disaster Management Datasets for Remote Sensing Based Emergency Applications](https://mlsanity.com/paper/2605.08196/) (2026)
- [SenseBench: A Benchmark for Remote Sensing Low-Level Visual Perception and Description in Large Vision-Language Models](https://mlsanity.com/paper/2605.10576/) (2026)
- [GeoDisaster: Benchmarking Orchestrated Agents for Operational Disaster Geo-Intelligence](https://mlsanity.com/paper/2606.17246/) (2026)
- [JL1-CC&QA: Extending the JL1-CD Benchmark with Change Captioning and Question Answering](https://mlsanity.com/paper/2606.31745/) (2026)
- [Uncertainty-aware tree height change regression](https://mlsanity.com/paper/2607.00638/) (2026)
- [GeoChrono: Benchmarking and Rethinking Long-Term Temporal Understanding in Remote Sensing](https://mlsanity.com/paper/2607.15768/) (2026)
- [More with Less: a Large Scale Remote Sensing VLM with a Simple Recipe](https://mlsanity.com/paper/2607.15942/) (2026)

## Abstract (arXiv)

Large vision-language models (VLMs) have made great achievements in Earth vision. However, complex disaster scenes with diverse disaster types, geographic regions, and satellite sensors have posed new challenges for VLM applications. To fill this gap, we curate a remote sensing vision-language dataset (DisasterM3) for global-scale disaster assessment and response. DisasterM3 includes 26,988 bi-temporal satellite images and 123k instruction pairs across 5 continents, with three characteristics: 1) Multi-hazard: DisasterM3 involves 36 historical disaster events with significant impacts, which are categorized into 10 common natural and man-made disasters. 2)Multi-sensor: Extreme weather during disasters often hinders optical sensor imaging, making it necessary to combine Synthetic Aperture Radar (SAR) imagery for post-disaster scenes. 3) Multi-task: Based on real-world scenarios, DisasterM3 includes 9 disaster-related visual perception and reasoning tasks, harnessing the full potential of VLM's reasoning ability with progressing from disaster-bearing body recognition to structural damage assessment and object relational reasoning, culminating in the generation of long-form disaster reports. We extensively evaluated 14 generic and remote sensing VLMs on our benchmark, revealing that state-of-the-art models struggle with the disaster tasks, largely due to the lack of a disaster-specific corpus, cross-sensor gap, and damage object counting insensitivity. Focusing on these issues, we fine-tune four VLMs using our dataset and achieve stable improvements across all tasks, with robust cross-sensor and cross-disaster generalization capabilities. The code and data are available at: https://github.com/Junjue-Wang/DisasterM3.