mlsanityData to 14 Oct 2026

Can Large Vision Language Models Read Maps Like a Human?

arXiv 2503.14607 ↗First posted 18 Mar 2025cs.CVcited by 30

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

In this paper, we introduce MapBench-the first dataset specifically designed for human-readable, pixel-based map-based outdoor navigation, curated from complex path finding scenarios. MapBench comprises over 1600 pixel space map path finding problems from 100 diverse maps. In MapBench, LVLMs generate language-based navigation instructions given a map image and a query with beginning and end landmarks. For each map, MapBench provides Map Space Scene Graph (MSSG) as an indexing data structure to convert between natural language and evaluate LVLM-generated results. We demonstrate that MapBench significantly challenges state-of-the-art LVLMs both zero-shot prompting and a Chain-of-Thought (CoT) augmented reasoning framework that decomposes map navigation into sequential cognitive processes. Our evaluation of both open-source and closed-source LVLMs underscores the substantial difficulty posed by MapBench, revealing critical limitations in their spatial reasoning and structured decision-making capabilities. We release all the code and dataset in https://github.com/taco-group/MapBench.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 73 references that are in the map. First posted on arXiv 18 Mar 2025.

Builds on

38 of 73 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answering8 Nov
  2. 2024Select2Plan: Training-Free ICL-Based Planning through VQA and Memory Retrieval6 Nov
  3. 2024Zero-shot Object Navigation with Vision-Language Models Reasoning24 Oct

Other references · 35

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  4. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  5. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  6. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  7. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  8. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  9. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  10. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  11. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
  12. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
23 more
  1. 2018Object Hallucination in Image Captioningcited by 669
  2. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  3. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
  4. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  5. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  6. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  7. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
  8. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
  9. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 294
  10. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 280
  11. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 231
  12. 2024GRAPE: Generalizing Robot Policy via Preference Alignmentcited by 109
  13. 2023Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillingscited by 69
  14. 2025MapNav: A Novel Memory Representation via Annotated Semantic Maps for Vision-and-Language Navigationcited by 60
  15. 2024AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Drivingcited by 35
  16. 2025Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimizationcited by 29
  17. 2024Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Modelscited by 27
  18. 2024Exploring Spatial Representation to Enhance LLM Reasoning in Aerial Vision-Language Navigationcited by 20
  19. 2024Video Quality Assessment: A Comprehensive Surveycited by 15
  20. 2023Vision and Language Navigation in the Real World via Online Visual Language Mappingcited by 11
  21. 2024Navigating Beyond Instructions: Vision-and-Language Navigation in Obstructed Environmentscited by 10
  22. 2024LOC-ZSON: Language-driven Object-Centric Zero-Shot Object Retrieval and Navigationcited by 8
  23. 2022Anticipating the Unseen Discrepancy for Vision and Language Navigationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 18; 2026: 12.

30 papers.

Sort:

By area:

  1. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 464
  2. 2025UniOcc: A Unified Benchmark for Occupancy Forecasting and Prediction in Autonomous Drivingcited by 28
  3. 2025SAFEFLOW: A Principled Protocol for Trustworthy and Transactional Autonomous Agent Systemscited by 25
  4. 2025ReasonMap: Towards Fine-Grained Visual Reasoning from Transit Mapscited by 23
  5. 2025RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via Multi-Stage Reinforcement Learningcited by 21
  6. 2025Embodied Web Agents: Bridging Physical-Digital Realms for Integrated Agent Intelligencecited by 15
  7. 2025Generative AI for Autonomous Driving: Frontiers and Opportunitiescited by 7
  8. 2025MapIQ: Evaluating Multimodal Large Language Models for Map Question Answeringcited by 5
  9. 2025Demystifying the Visual Quality Paradox in Multimodal Large Language Modelscited by 3
  10. 2025RDD: Retrieval-Based Demonstration Decomposer for Planner Alignment in Long-Horizon Taskscited by 3
  11. 2025FRIEDA: Benchmarking Multi-Step Cartographic Reasoning in Vision-Language Modelscited by 3
  12. 2025MapTrace: Scalable Data Generation for Route Tracing on Mapscited by 3
18 more
  1. 2026MapTab: A Diagnostic Benchmark for Long-Horizon Multi-Criteria Multimodal Reasoning on Heterogeneous Topological Graphscited by 3
  2. 2025The Role of Open-Source LLMs in Shaping the Future of GeoAIcited by 2
  3. 2025VLM@school -- Evaluation of AI image understanding on German middle school knowledgecited by 2
  4. 2025Q-Router: Agentic Video Quality Assessment with Expert Model Routing and Artifact Localizationcited by 2
  5. 2026OmniMapBench: Benchmarking Visual-Centric Reasoning on Diverse Map Documentscited by 2
  6. 2025Does RLVR Extend Reasoning Boundaries? Investigating Capability Expansion in Vision-Language Modelscited by 1
  7. 2026Mil-SCORE: Benchmarking Long-Context Geospatial Reasoning and Planning in Large Language Modelscited by 1
  8. 2026TraversalBench: Challenging Paths to Follow for Vision Language Modelscited by 1
  9. 2026360CityArena: A Realistic Virtual Urban Navigation Benchmark for Embodied Agentscited by 1
  10. 2026ICDAR2026 Competition on Multimodal Reasoning over Documents in Multiple Domainscited by 1
  11. 2025SVG Decomposition for Enhancing Large Multimodal Models Visualization Comprehension: A Study with Floor Planscited by 0
  12. 2025LAVQA: A Latency-Aware Visual Question Answering Framework for Shared Autonomy in Self-Driving Vehiclescited by 0
  13. 2026TransitLM: A Large-Scale Dataset and Benchmark for Map-Free Transit Route Generationcited by 0
  14. 2026PGT: Procedurally Generated Tasks for improving visual grounding in MLLMscited by 0
  15. 2026Perceive, Interact, Reason: Building Tool-Augmented Visual Agents for Spatial Reasoningcited by 0
  16. 2026MapReason-OSM: Can Vision-Language Models Make Graph-Verifiable Mobility Decisions from Street Maps ?cited by 0
  17. 2026Toward AI-Friendly Cartography: Understanding How Color Design Influences Foundation Model Spatial Reasoning on Sequential Choropleth Mapscited by 0
  18. 2026AirLog: Store-Level Indoor Life Logging Made Easycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026m2sv: A Scalable Benchmark for Map-to-Street-View Spatial Reasoning
  2. 2024Vision Language Models Can Parse Floor Plan Maps
  3. 2025CartoMapQA: A Fundamental Benchmark Dataset Evaluating Vision-Language Models on Cartographic Map Understanding
  4. 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
  5. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
  6. 2025RoadBench: Benchmarking MLLMs on Fine-Grained Spatial Understanding and Reasoning under Urban Road Scenarios
  7. 2026PlanBench-V: A Spatial Planning Map Benchmark for Vision-Language Models
  8. 2026Mind the Spatial Gap: A Benchmark for Geospatial Reasoning and Constrained Spatial Editing