mlsanityData to 7 Oct 2026

Spatial Competence Benchmark

arXiv 2604.09594 ↗First posted 5 Mar 2026cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Spatial competence is the quality of maintaining a consistent internal representation of an environment and using it to infer discrete structure and plan actions under constraints. Prevailing spatial evaluations for large models are limited to probing isolated primitives through 3D transformations or visual question answering. We introduce the Spatial Competence Benchmark (SCBench), spanning three hierarchical capability buckets whose tasks require executable outputs verified by deterministic checkers or simulator-based evaluators. On SCBench, three frontier models exhibit monotonically decreasing accuracy up the capability ladder. Sweeping output-token caps shows that accuracy gains concentrate at low budgets and saturate quickly, and failures are dominated by locally plausible geometry that breaks global constraints. We release the task generators, verifiers, and visualisation tooling.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 16 references that are in the map. First posted on arXiv 5 Mar 2026.

Builds on

12 of 16 references.

References · 12

  1. 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Modelscited by 111
  2. 2025lmgame-Bench: How Good are LLMs at Playing Games?cited by 54
  3. 2025LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning?cited by 49
  4. 2025Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulationscited by 21
  5. 2025When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thoughtcited by 18
  6. 2025FloorplanQA: A Benchmark for Spatial Reasoning in LLMs using Structured Representationscited by 13
  7. 2025SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMscited by 12
  8. 2024Probing Mechanical Reasoning in Large Vision Language Modelscited by 9
  9. 202511Plus-Bench: Demystifying Multimodal LLM Spatial Reasoning with Cognitive-Inspired Analysiscited by 8
  10. 2026Can LLMs See Without Pixels? Benchmarking Spatial Intelligence from Textual Descriptionscited by 7
  11. 2025GamiBench: Evaluating Spatial Reasoning and 2D-to-3D Planning Capabilities of MLLMs with Origami Folding Taskscited by 4
  12. 2025APEX: Empowering LLMs with Physics-Based Task Planning for Real-time Insightcited by 1

Built on by

Not cited yet.