mlsanityData to 14 Oct 2026

11Plus-Bench: Demystifying Multimodal LLM Spatial Reasoning with Cognitive-Inspired Analysis

arXiv 2508.20068 ↗First posted 27 Aug 2025cs.CL cs.CV cs.LGcited by 9

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more
Abstract (arXiv)

For human cognitive process, spatial reasoning and perception are closely entangled, yet the nature of this interplay remains underexplored in the evaluation of multimodal large language models (MLLMs). While recent MLLM advancements show impressive performance on reasoning, their capacity for human-like spatial cognition remains an open question. In this work, we introduce a systematic evaluation framework to assess the spatial reasoning abilities of state-of-the-art MLLMs relative to human performance. Central to our work is 11Plus-Bench, a high-quality benchmark derived from realistic standardized spatial aptitude tests. 11Plus-Bench also features fine-grained expert annotations of both perceptual complexity and reasoning process, enabling detailed instance-level analysis of model behavior. Through extensive experiments across 14 MLLMs and human evaluation, we find that current MLLMs exhibit early signs of spatial cognition. Despite a large performance gap compared to humans, MLLMs' cognitive profiles resemble those of humans in that cognitive effort correlates strongly with reasoning-related complexity. However, instance-level performance in MLLMs remains largely random, whereas human correctness is highly predictable and shaped by abstract pattern complexity. These findings highlight both emerging capabilities and limitations in current MLLMs' spatial reasoning capabilities and provide actionable insights for advancing model design.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 83 references that are in the map. First posted on arXiv 27 Aug 2025.

Builds on

20 of 83 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Visual Planning: Let's Think Only with Images16 May
  2. 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipes21 Apr
  3. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models25 Mar
  4. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec

Other references · 16

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2024GPT-4o System Cardcited by 4,588
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  6. 2025Gemma 3 Technical Reportcited by 1,824
  7. 2023PaLM 2 Technical Reportcited by 1,162
  8. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 206
  9. 2025ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systemscited by 190
  10. 2024ARC Prize 2024: Technical Reportcited by 104
  11. 2025General Scales Unlock AI Evaluation with Explanatory and Predictive Powercited by 31
  12. 2025Defining and Evaluating Visual Language Models' Basic Spatial Abilities: A Perspective from Psychometricscited by 23
4 more
  1. 2023Inferring Capabilities from Task Performance with Bayesian Triangulationcited by 13
  2. 2023Predictable Artificial Intelligencecited by 8
  3. 2024Wing Optimisation for a tractor propeller driven Micro Aerial Vehiclecited by 4
  4. 2024Entanglement Routing in Quantum Networks: A Comprehensive Surveycited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 7.

9 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods
  2. 2025SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognition
  3. 2026From Human Cognition to Neural Activations: Probing the Computational Primitives of Spatial Reasoning in LLMs
  4. 2026From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs
  5. 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
  6. 2025Imagine in Space: Exploring the Frontier of Spatial Intelligence and Reasoning Efficiency in Vision Language Models
  7. 2025Spatial4D-Bench: A Versatile 4D Spatial Intelligence Benchmark
  8. 2026Towards a Data-Centric Understanding of Spatial Reasoning in Multimodal Large Language Models