# DisasterVQA: A Visual Question Answering Benchmark Dataset for Disaster Scenes

arXiv: https://arxiv.org/abs/2601.13839 · HTML: https://mlsanity.com/paper/2601.13839/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 15 of its 49 references that are in the map.
- First posted on arXiv 20 Jan 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 15 of 15)

- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities](https://mlsanity.com/paper/2308.02490/) (2023)
- [Visual7W: Grounded Question Answering in Images](https://mlsanity.com/paper/1511.03416/) (2015)
- [RSVQA: Visual Question Answering for Remote Sensing Data](https://mlsanity.com/paper/2003.07333/) (2020)
- [FloodNet: A High Resolution Aerial Imagery Dataset for Post Flood Scene Understanding](https://mlsanity.com/paper/2012.02951/) (2020)
- [CrisisMMD: Multimodal Twitter Datasets from Natural Disasters](https://mlsanity.com/paper/1805.00713/) (2018)
- [EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialogues](https://mlsanity.com/paper/2412.15190/) (2024)
- [GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks](https://mlsanity.com/paper/2411.19325/) (2024)
- [UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling](https://mlsanity.com/paper/2408.04810/) (2024)
- [Incidents1M: a large-scale dataset of images with natural disasters, damage, and incidents](https://mlsanity.com/paper/2201.04236/) (2022)
- [MEDIC: A Multi-Task Learning Dataset for Disaster Image Classification](https://mlsanity.com/paper/2108.12828/) (2021)
- [Visual Question Answering on Multiple Remote Sensing Image Modalities](https://mlsanity.com/paper/2505.15401/) (2025)
- [ZeShot-VQA: Zero-Shot Visual Question Answering Framework with Answer Mapping for Natural Disaster Damage Assessment](https://mlsanity.com/paper/2506.00238/) (2025)
- [Reducing Hallucinations: Enhancing VQA for Flood Disaster Damage Assessment with Visual Contexts](https://mlsanity.com/paper/2312.13848/) (2023)

## Built on by (most cited first, 1 of 1)

- [GeoRisk-RAG: A Hierarchy-Aware Risk Framework for Improving RAG Reliability through Selective Answering](https://mlsanity.com/paper/2608.22634/) (2026)

## Abstract (arXiv)

Social media imagery provides a low-latency source of situational information during natural and human-induced disasters, enabling rapid damage assessment and response. While Visual Question Answering (VQA) has shown strong performance in general-purpose domains, its suitability for the complex and safety-critical reasoning required in disaster response remains unclear. We introduce DisasterVQA, a benchmark dataset designed for perception and reasoning in crisis contexts. DisasterVQA consists of 1,395 real-world images and 4,405 expert-curated question-answer pairs spanning diverse events such as floods, wildfires, and earthquakes. Grounded in humanitarian frameworks including FEMA ESF and OCHA MIRA, the dataset includes binary, multiple-choice, and open-ended questions covering situational awareness and operational decision-making tasks. We benchmark seven state-of-the-art vision-language models and find performance variability across question types, disaster categories, regions, and humanitarian tasks. Although models achieve high accuracy on binary questions, they struggle with fine-grained quantitative reasoning, object counting, and context-sensitive interpretation, particularly for underrepresented disaster scenarios. DisasterVQA provides a challenging and practical benchmark to guide the development of more robust and operationally meaningful vision-language models for disaster response. The dataset is publicly available at https://doi.org/10.5281/zenodo.18267769.