MAPWise: Evaluating Vision-Language Models for Advanced Map Queries
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
Vision-language models (VLMs) excel at tasks requiring joint understanding of visual and linguistic information. A particularly promising yet under-explored application for these models lies in answering questions based on various kinds of maps. This study investigates the efficacy of VLMs in answering questions based on choropleth maps, which are widely used for data analysis and representation. To facilitate and encourage research in this area, we introduce a novel map-based question-answering benchmark, consisting of maps from three geographical regions (United States, India, China), each containing 1000 questions. Our benchmark incorporates 43 diverse question templates, requiring nuanced understanding of relative spatial relationships, intricate map features, and complex reasoning. It also includes maps with discrete and continuous values, encompassing variations in color-mapping, category ordering, and stylistic patterns, enabling comprehensive analysis. We evaluate the performance of multiple VLMs on this benchmark, highlighting gaps in their abilities and providing insights for improving such models.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 23 references that are in the map. First posted on arXiv 30 Aug 2024.
Builds on
13 of 23 references.
Closest ancestors · 1
Same direction, newest first
Other references · 12
- 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
- 2015VQA: Visual Question Answeringcited by 2,489
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
- 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
- 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 338
- 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
- 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 314
- 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 121
- 2022MapQA: A Dataset for Question Answering on Choropleth Mapscited by 82
- 2019LEAF-QA: Locate, Encode & Attend for Figure Question Answeringcited by 53
- 2021Visual Question Rewriting for Increasing Response Ratecited by 3
- 2024Understanding Reader Takeaways in Thematic Maps Under Varying Text, Detail, and Spatial Autocorrelationcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025MapIQ: Evaluating Multimodal Large Language Models for Map Question Answering
- 2026MapVerse: A Benchmark for Geospatial Question Answering on Diverse Real-World Maps
- 2025CartoMapQA: A Fundamental Benchmark Dataset Evaluating Vision-Language Models on Cartographic Map Understanding
- 2026OmniMapBench: Benchmarking Visual-Centric Reasoning on Diverse Map Documents
- 2025ReasonMap: Towards Fine-Grained Visual Reasoning from Transit Maps
- 2026SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models
- 2024MapQaTor: An Extensible Framework for Efficient Annotation of Map-Based QA Datasets
- 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey