mlsanityData to 14 Oct 2026

ReasonMap: Towards Fine-Grained Visual Reasoning from Transit Maps

arXiv 2505.18675 ↗First posted 24 May 2025cs.CV cs.AI cs.CLcited by 23

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Multimodal large language models (MLLMs) have demonstrated significant progress in semantic scene understanding and text-image alignment, with reasoning variants enhancing performance on more complex tasks involving mathematics and logic. To bridge this gap, we introduce ReasonMap, a novel benchmark specifically designed to evaluate these capabilities. ReasonMap encompasses high-resolution transit maps from 30 cities and includes 1,008 question-answer pairs spanning two question types and three templates. Furthermore, we design a two-level evaluation pipeline that properly assesses answer correctness and quality. Our comprehensive evaluation of 16 popular MLLMs reveals a counterintuitive pattern: among open-source models, base variants outperform their reasoning-tuned counterparts, whereas the opposite trend is observed in closed-source models. Further analysis under the visual-masking setting confirms that strong performance necessitates direct visual grounding, rather than relying solely on language priors. We further establish a training baseline with reinforcement fine-tuning, providing a reference for future exploration. We hope this benchmark study offers new insights into visual reasoning and helps investigate the gap between open- and closed-source models.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 42 of its 102 references that are in the map. First posted on arXiv 24 May 2025.

Builds on

42 of 102 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025R-Bench: Graduate-level Multi-disciplinary Benchmarks for LLM & MLLM Complex Reasoning Evaluation4 May
  2. 2025Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning23 Apr
  3. 2025VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models21 Apr
  4. 2025VisualPuzzles: Decoupling Multimodal Reasoning Evaluation from Domain Knowledge14 Apr
  5. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
  6. 2025Kimi-VL Technical Report10 Apr
  7. 2025Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought8 Apr
  8. 2025VGRP-Bench: Visual Grid Reasoning Puzzle Benchmark for Large Vision-Language Models29 Mar
  9. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thought13 Jan
  10. 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark9 Jan
  11. 2024Visual-Linguistic Agent: Towards Collaborative Contextual Object Reasoning15 Nov
  12. 2024MathGLM-Vision: Solving Mathematical Problems with Multi-Modal Large Language Model10 Sep

Other references · 30

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2024Qwen2 Technical Reportcited by 2,120
  7. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  8. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  9. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  10. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
  11. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
  12. 2025Seed1.5-VL Technical Reportcited by 373
18 more
  1. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 295
  2. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoningcited by 78
  3. 2024An Empirical Analysis on Spatial Reasoning Capabilities of Large Multimodal Modelscited by 50
  4. 2025Med-RLVR: Emerging Medical Reasoning from a 3B base model via reinforcement Learningcited by 40
  5. 2024PlanAgent: A Multi-modal Large Language Agent for Closed-loop Vehicle Motion Planningcited by 38
  6. 2025Can Large Vision Language Models Read Maps Like a Human?cited by 30
  7. 2024V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encodingcited by 28
  8. 2024CityBench: Evaluating the Capabilities of Large Language Models for Urban Taskscited by 27
  9. 2024Perceive, Reflect, and Plan: Designing LLM Agent for Goal-Directed City Navigation without Instructionscited by 25
  10. 2025SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained Understandingcited by 25
  11. 2024MapEval: A Map-Based Evaluation of Geo-Spatial Reasoning in Foundation Modelscited by 21
  12. 2025GeoNav: Empowering MLLMs with dual-scale geospatial reasoning for language-goal aerial navigationcited by 16
  13. 2025AI Sees Your Location, But With A Bias Toward The Wealthy Worldcited by 14
  14. 2024Is Oracle Pruning the True Oracle?cited by 7
  15. 2016Learning from Maps: Visual Common Sense for Autonomous Drivingcited by 4
  16. 2024Understanding the Role of LLMs in Multimodal Evaluation Benchmarkscited by 4
  17. 2025Perceiving Beyond Language Priors: Enhancing Visual Comprehension and Attention in Multimodal Modelscited by 4
  18. 2025Instruction-augmented Multimodal Alignment for Image-Text and Element Matchingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 12.

23 papers.

Sort:

By area:

  1. 2025Semantic-Aware Logical Reasoning via a Semiotic Frameworkcited by 32
  2. 2025A Survey of Token Compression for Efficient Multimodal Large Language Modelscited by 25
  3. 2025RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via Multi-Stage Reinforcement Learningcited by 21
  4. 2025OBS-Diff: Accurate Pruning For Diffusion Models in One-Shotcited by 19
  5. 2025Active Perception Agent for Omnimodal Audio-Video Understandingcited by 13
  6. 2025Revisiting the Necessity of Lengthy Chain-of-Thought in Vision-centric Reasoning Generalizationcited by 8
  7. 2025MergeMix: A Unified Augmentation Paradigm for Visual and Multi-Modal Understandingcited by 7
  8. 2025Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systemscited by 4
  9. 2025FRIEDA: Benchmarking Multi-Step Cartographic Reasoning in Vision-Language Modelscited by 3
  10. 2026MapTab: A Diagnostic Benchmark for Long-Horizon Multi-Criteria Multimodal Reasoning on Heterogeneous Topological Graphscited by 3
  11. 2026Self-Purification Mitigates Backdoors in Multimodal Diffusion Language Modelscited by 2
  12. 2026OmniMapBench: Benchmarking Visual-Centric Reasoning on Diverse Map Documentscited by 2
11 more
  1. 2025GTR-Bench: Evaluating Geo-Temporal Reasoning in Vision-Language Modelscited by 1
  2. 2025Does RLVR Extend Reasoning Boundaries? Investigating Capability Expansion in Vision-Language Modelscited by 1
  3. 2026CLBench-V: Evaluating Multimodal Context Learning from Grounding to Knowledge Acquisitioncited by 1
  4. 2026EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memorycited by 1
  5. 2026PGT: Procedurally Generated Tasks for improving visual grounding in MLLMscited by 0
  6. 2026VisualNeedle: Benchmarking Active Visual Search in Information-Dense Scenescited by 0
  7. 2026UltraVR: A Diagnostic Ultra-Resolution Image-VQA Benchmark for Evidence-Grounded Reasoningcited by 0
  8. 2026Perceive, Interact, Reason: Building Tool-Augmented Visual Agents for Spatial Reasoningcited by 0
  9. 2026LabInstruct: Benchmarking Situated Instructional Video Generation for Lab Procedurescited by 0
  10. 2026VIEWCRAFT: Evolving What Multimodal Agents See and How They Read Itcited by 0
  11. 2026Game2Real-Bench: Do Toy Games Reveal and Transfer to Real-World Visual Reasoning?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts
  2. 2025ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models
  3. 2026LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models
  4. 2024From the Least to the Most: Building a Plug-and-Play Visual Reasoner via Data Synthesis
  5. 2025VGR: Visual Grounded Reasoning
  6. 2026When Prompts Become Pixels: Prompt-Region Grounding for Multimodal Reasoning
  7. 2026MapVerse: A Benchmark for Geospatial Question Answering on Diverse Real-World Maps
  8. 2025GeoChain: Multimodal Chain-of-Thought for Geographic Reasoning