# MapIQ: Evaluating Multimodal Large Language Models for Map Question Answering

arXiv: https://arxiv.org/abs/2507.11625 · HTML: https://mlsanity.com/paper/2507.11625/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 19 of its 48 references that are in the map.
- First posted on arXiv 15 Jul 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 19 of 19)

- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [DVQA: Understanding Data Visualizations via Question Answering](https://mlsanity.com/paper/1801.08163/) (2018)
- [LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding](https://mlsanity.com/paper/2306.17107/) (2023)
- [Building and better understanding vision-language models: insights and future directions](https://mlsanity.com/paper/2408.12637/) (2024)
- [A Survey of Large Language Models in Medicine: Progress, Application, and Challenge](https://mlsanity.com/paper/2311.05112/) (2023)
- [MapQA: A Dataset for Question Answering on Choropleth Maps](https://mlsanity.com/paper/2211.08545/) (2022)
- [ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answering](https://mlsanity.com/paper/2405.07001/) (2024)
- [Charting New Territories: Exploring the Geographic and Geospatial Capabilities of Multimodal LLMs](https://mlsanity.com/paper/2311.14656/) (2023)
- [Can Large Vision Language Models Read Maps Like a Human?](https://mlsanity.com/paper/2503.14607/) (2025)
- [Exploring the Capability of LLMs in Performing Low-Level Visual Analytic Tasks on SVG Data Visualizations](https://mlsanity.com/paper/2404.19097/) (2024)
- [Unraveling the Truth: Do VLMs really Understand Charts? A Deep Dive into Consistency and Robustness](https://mlsanity.com/paper/2407.11229/) (2024)
- [Mini-VLAT: A Short and Effective Measure of Visualization Literacy](https://mlsanity.com/paper/2304.07905/) (2023)
- [Correctness Comparison of ChatGPT-4, Gemini, Claude-3, and Copilot for Spatial Tasks](https://mlsanity.com/paper/2401.02404/) (2024)
- [Understanding Reader Takeaways in Thematic Maps Under Varying Text, Detail, and Spatial Autocorrelation](https://mlsanity.com/paper/2403.08260/) (2024)

## Built on by (most cited first, 5 of 5)

- [MapTab: A Diagnostic Benchmark for Long-Horizon Multi-Criteria Multimodal Reasoning on Heterogeneous Topological Graphs](https://mlsanity.com/paper/2602.18600/) (2026)
- [ENC-Bench: A Benchmark for Evaluating Multimodal Large Language Models in Electronic Navigational Chart Understanding](https://mlsanity.com/paper/2603.22763/) (2026)
- [MapVerse: A Benchmark for Geospatial Question Answering on Diverse Real-World Maps](https://mlsanity.com/paper/2602.10518/) (2026)
- [TimeSpot: Benchmarking Geo-Temporal Understanding in Vision-Language Models in Real-World Settings](https://mlsanity.com/paper/2603.06687/) (2026)
- [DRAGON: A Benchmark for Evidence-Grounded Visual Reasoning over Diagrams](https://mlsanity.com/paper/2604.25231/) (2026)

## Abstract (arXiv)

Recent advancements in multimodal large language models (MLLMs) have driven researchers to explore how well these models read data visualizations, e.g., bar charts, scatter plots. More recently, attention has shifted to visual question answering with maps (Map-VQA). However, Map-VQA research has primarily focused on choropleth maps, which cover only a limited range of thematic categories and visual analytical tasks. To address these gaps, we introduce MapIQ, a benchmark dataset comprising 14,706 question-answer pairs across three map types: choropleth maps, cartograms, and proportional symbol maps spanning topics from six distinct themes (e.g., housing, crime). We evaluate multiple MLLMs using six visual analytical tasks, comparing their performance against one another and a human baseline. An additional experiment examining the impact of map design changes (e.g., altered color schemes, modified legend designs, and removal of map elements) provides insights into the robustness and sensitivity of MLLMs, their reliance on internal geographic knowledge, and potential avenues for improving Map-VQA performance.