mlsanityData to 7 Oct 2026

ArchSIBench: Benchmarking the Architectural Spatial Intelligence of Vision-Language Models

arXiv 2605.20837 ↗First posted 20 May 2026cs.CV cs.AIcited by 2

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Architectural spatial intelligence, the ability to recognize and infer architectural space, is fundamental to tasks such as robot navigation, embodied interaction, and 3D scene understanding and generation. Although extensive research has evaluated the basic spatial skills of Vision-Language Models (VLMs) such as relative orientation, distance comparison, and object counting, these tasks cover only the most elementary levels of spatial cognition and largely overlook higher-level cognition of architectural space, including layout understanding, circulation patterns, and functional zoning. In this work, we present ArchSIBench, a Benchmark for Architectural Spatial Intelligence based on the perspectives from architecture, cognitive science, and psychology. ArchSIBench covers five core dimensions: perception, reasoning, navigation, transformation, and configuration, comprising 17 fine-grained subtasks. Through careful manual annotation by experts with architectural backgrounds, we construct 3,000 question-answer pairs to enable comprehensive evaluation of architectural spatial intelligence. Based on ArchSIBench, we evaluate various VLMs and find that the architectural spatial intelligence of most models shows significant differences from human baselines; additionally, models exhibit substantial variability across capability dimensions. Some state-of-the-art models can approach the level of human evaluators without architectural training. However, a clear gap remains compared to human evaluators with architectural training, particularly in spatial transformation and configuration reasoning. We believe that ArchSIBench will provide important insights and systematic resources for measuring and advancing the architectural spatial intelligence of VLMs. The dataset and code are available at https://huggingface.co/datasets/ArchSIBench/ArchSIBench.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 74 references that are in the map. First posted on arXiv 20 May 2026.

Builds on

20 of 74 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods14 Nov
  2. 2025Open3D-VQA: A Benchmark for Comprehensive Spatial Reasoning with Multimodal Large Language Model in Open Space14 Mar

Other references · 18

  1. 2024GPT-4o System Cardcited by 4,467
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  3. 2025Qwen3-VL Technical Reportcited by 2,534
  4. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  5. 2025OpenAI GPT-5 System Cardcited by 910
  6. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
  7. 2024An Introduction to Vision-Language Modelingcited by 175
  8. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 170
  9. 2022A Review of Sparse Expert Models in Deep Learningcited by 147
  10. 2025Scenethesis: A Language and Vision Agentic Framework for 3D Scene Generationcited by 65
  11. 2025SpatialLM: Training Large Language Models for Structured Indoor Modelingcited by 63
  12. 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challengescited by 46
6 more
  1. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 31
  2. 2025SPATIALGEN: Layout-guided 3D Indoor Scene Generationcited by 14
  3. 2026AECV-Bench: Benchmarking Multimodal Models on Architectural and Engineering Drawings Understandingcited by 8
  4. 2025Blueprint-Bench: Comparing spatial intelligence of LLMs, agents and image modelscited by 5
  5. 2026Spatial-VLN: Zero-Shot Vision-and-Language Navigation With Explicit Spatial Perception and Explorationcited by 4
  6. 2025FloorPlan-DeepSeek (FPDS): A multimodal approach to floorplan generation using vector-based next room predictioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026CrossProjection: Geometric Grounding Beyond Viewpoint Change in Architectural Drawingscited by 0
  2. 2026ARCH-B: Architectural Representation, Comprehension and Hierarchy Benchmarkcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognition
  2. 2025Defining and Evaluating Visual Language Models' Basic Spatial Abilities: A Perspective from Psychometrics
  3. 2026Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces
  4. 2025Imagine in Space: Exploring the Frontier of Spatial Intelligence and Reasoning Efficiency in Vision Language Models
  5. 2026From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs
  6. 2026Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Tests
  7. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective
  8. 2026KilometerVision: A New Frontier for Large-Scale Spatial Intelligence in VLMs