11Plus-Bench: Demystifying Multimodal LLM Spatial Reasoning with Cognitive-Inspired Analysis
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
For human cognitive process, spatial reasoning and perception are closely entangled, yet the nature of this interplay remains underexplored in the evaluation of multimodal large language models (MLLMs). While recent MLLM advancements show impressive performance on reasoning, their capacity for human-like spatial cognition remains an open question. In this work, we introduce a systematic evaluation framework to assess the spatial reasoning abilities of state-of-the-art MLLMs relative to human performance. Central to our work is 11Plus-Bench, a high-quality benchmark derived from realistic standardized spatial aptitude tests. 11Plus-Bench also features fine-grained expert annotations of both perceptual complexity and reasoning process, enabling detailed instance-level analysis of model behavior. Through extensive experiments across 14 MLLMs and human evaluation, we find that current MLLMs exhibit early signs of spatial cognition. Despite a large performance gap compared to humans, MLLMs' cognitive profiles resemble those of humans in that cognitive effort correlates strongly with reasoning-related complexity. However, instance-level performance in MLLMs remains largely random, whereas human correctness is highly predictable and shaped by abstract pattern complexity. These findings highlight both emerging capabilities and limitations in current MLLMs' spatial reasoning capabilities and provide actionable insights for advancing model design.
Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 83 references that are in the map. First posted on arXiv 27 Aug 2025.
Builds on
20 of 83 references.
Closest ancestors · 4
Same direction, newest first
- 2025Visual Planning: Let's Think Only with Images16 May
- 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipes21 Apr
- 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models25 Mar
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
Other references · 16
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2025Gemma 3 Technical Reportcited by 1,824
- 2023PaLM 2 Technical Reportcited by 1,162
- 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 206
- 2025ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systemscited by 190
- 2024ARC Prize 2024: Technical Reportcited by 104
- 2025General Scales Unlock AI Evaluation with Explanatory and Predictive Powercited by 31
- 2025Defining and Evaluating Visual Language Models' Basic Spatial Abilities: A Perspective from Psychometricscited by 23
4 more
Built on by
Citing papers per year: 2025: 2; 2026: 7.
9 papers.
- 2025Latent Sketchpad: Sketching Visual Thoughts to Elicit Multimodal Reasoning in MLLMscited by 43
- 2026How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editingcited by 12
- 2026Thinking in Frames: How Visual Context and Test-Time Scaling Empower Video Reasoningcited by 10
- 2026Multimodal Language Models Cannot Spot Spatial Inconsistenciescited by 2
- 2025Evaluating Multimodal Large Language Models with Daily Composite Tasks in Home Environmentscited by 1
- 2026STEMVerse: A Dual-Axis Diagnostic Framework for STEM Reasoning in Large Language Modelscited by 1
- 2026Do multimodal models imagine electric sheep?cited by 1
- 2026PointVG-R: Internalizing Geometric Reasoning in MLLMs for Precise Pointing Localization via Visual Chain of Thoughtcited by 1
- 2026Spatial Competence Benchmarkcited by 0
- 2026PointVG-R: Internalizing Geometric Reasoning in MLLMs for Precise Pointing Localization via Visual Chain of Thought23 Jun 2026
- 2026Do multimodal models imagine electric sheep?10 May 2026
- 2026Multimodal Language Models Cannot Spot Spatial Inconsistencies1 Apr 2026
- 2026Spatial Competence Benchmark5 Mar 2026
- 2026How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editing2 Feb 2026
- 2026Thinking in Frames: How Visual Context and Test-Time Scaling Empower Video Reasoning28 Jan 2026
- 2026STEMVerse: A Dual-Axis Diagnostic Framework for STEM Reasoning in Large Language Models14 Jan 2026
- 2025Latent Sketchpad: Sketching Visual Thoughts to Elicit Multimodal Reasoning in MLLMs28 Oct 2025
- 2025Evaluating Multimodal Large Language Models with Daily Composite Tasks in Home Environments22 Sep 2025
Similar, not cited
Close in wording, no citation either way
- 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods
- 2025SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognition
- 2026From Human Cognition to Neural Activations: Probing the Computational Primitives of Spatial Reasoning in LLMs
- 2026From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs
- 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
- 2025Imagine in Space: Exploring the Frontier of Spatial Intelligence and Reasoning Efficiency in Vision Language Models
- 2025Spatial4D-Bench: A Versatile 4D Spatial Intelligence Benchmark
- 2026Towards a Data-Centric Understanding of Spatial Reasoning in Multimodal Large Language Models