# iVISPAR -- An Interactive Visual-Spatial Reasoning Benchmark for VLMs

arXiv: https://arxiv.org/abs/2502.03214 · HTML: https://mlsanity.com/paper/2502.03214/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 42 of its 57 references that are in the map.
- First posted on arXiv 5 Feb 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)
- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 30 of 42)

- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models](https://mlsanity.com/paper/2406.01584/) (2024)
- [Visual Spatial Reasoning](https://mlsanity.com/paper/2205.00363/) (2022)
- [What's "up" with vision-language models? Investigating their struggle with spatial reasoning](https://mlsanity.com/paper/2310.19785/) (2023)
- [An Introduction to Vision-Language Modeling](https://mlsanity.com/paper/2405.17247/) (2024)
- [Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planning](https://mlsanity.com/paper/2311.17842/) (2023)
- [Linearly Mapping from Image to Text Space](https://mlsanity.com/paper/2209.15162/) (2022)
- [Large Language Models for Robotics: A Survey](https://mlsanity.com/paper/2311.07226/) (2023)
- [Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color](https://mlsanity.com/paper/2109.06129/) (2021)
- [Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models](https://mlsanity.com/paper/2406.14852/) (2024)
- [Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoning](https://mlsanity.com/paper/2212.00259/) (2022)
- [SpartQA: : A Textual Question Answering Benchmark for Spatial Reasoning](https://mlsanity.com/paper/2104.05832/) (2021)
- [Understanding the Limits of Vision Language Models Through the Lens of the Binding Problem](https://mlsanity.com/paper/2411.00238/) (2024)
- [Evaluating Spatial Understanding of Large Language Models](https://mlsanity.com/paper/2310.14540/) (2023)
- [Can Large Language Models be Good Path Planners? A Benchmark and Investigation on Spatial-temporal Reasoning](https://mlsanity.com/paper/2310.03249/) (2023)
- [VSP: Assessing the dual challenges of perception and reasoning in spatial planning tasks for VLMs](https://mlsanity.com/paper/2407.01863/) (2024)
- [Learning Language Games through Interaction](https://mlsanity.com/paper/1606.02447/) (2016)
- [ShapeWorld - A new test methodology for multimodal language understanding](https://mlsanity.com/paper/1704.04517/) (2017)
- [Sparkle: Mastering Basic Spatial Capabilities in Vision Language Models Elicits Generalization to Spatial Reasoning](https://mlsanity.com/paper/2410.16162/) (2024)
- [GSR-BENCH: A Benchmark for Grounded Spatial Reasoning Evaluation via Multimodal LLMs](https://mlsanity.com/paper/2406.13246/) (2024)
- [GRASP: A novel benchmark for evaluating language GRounding And Situated Physics understanding in multimodal language models](https://mlsanity.com/paper/2311.09048/) (2023)
- [Benchmarking Vision, Language, & Action Models on Robotic Learning Tasks](https://mlsanity.com/paper/2411.05821/) (2024)

## Built on by (most cited first, 25 of 25)

- [MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMs](https://mlsanity.com/paper/2503.13111/) (2025)
- [Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models](https://mlsanity.com/paper/2504.21277/) (2025)
- [SD-VLM: Spatial Measuring and Understanding with Depth-Encoded Vision-Language Models](https://mlsanity.com/paper/2509.17664/) (2025)
- [Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipes](https://mlsanity.com/paper/2504.15037/) (2025)
- [STAR-R1: Spatial TrAnsformation Reasoning by Reinforcing Multimodal LLMs](https://mlsanity.com/paper/2505.15804/) (2025)
- [Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Models](https://mlsanity.com/paper/2505.18536/) (2025)
- [SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environments](https://mlsanity.com/paper/2604.14144/) (2026)
- [MIRAGE: A Multi-modal Benchmark for Spatial Perception, Reasoning, and Intelligence](https://mlsanity.com/paper/2505.10604/) (2025)
- [Pixels to Principles: Probing Intuitive Physics Understanding in Multimodal Language Models](https://mlsanity.com/paper/2507.16572/) (2025)
- [Can Vision-Language Models See Squares? Text-Recognition Mediates Spatial Reasoning Across Three Model Families](https://mlsanity.com/paper/2602.15950/) (2026)
- [Do Vision-Language-Models show human-like logical problem-solving capability in point and click puzzle games?](https://mlsanity.com/paper/2605.11223/) (2026)
- [PRISM: A Benchmark for Programmatic Spatial-Temporal Reasoning](https://mlsanity.com/paper/2605.19382/) (2026)
- [Aggregating LLM-Based Weak Verifiers for Spatial Layout Generation](https://mlsanity.com/paper/2606.05268/) (2026)
- [UrbanGround: From Local Perception to Spatial Agency in a Real-Scale City](https://mlsanity.com/paper/2608.27456/) (2026)
- [Ascending the Infinite Ladder: Benchmarking Spatial Deformation Reasoning in Vision-Language Models](https://mlsanity.com/paper/2507.02978/) (2025)
- [Can you SPLICE it together? A Human Curated Benchmark for Probing Visual Reasoning in VLMs](https://mlsanity.com/paper/2509.24640/) (2025)
- [A Multi-Modal Neuro-Symbolic Approach for Spatial Reasoning-Based Visual Grounding in Robotics](https://mlsanity.com/paper/2510.27033/) (2025)
- [Grid Spatial Understanding: A Dataset for Textual Spatial Reasoning over Grids, Embodied Settings, and Coordinate Structures](https://mlsanity.com/paper/2603.17333/) (2026)
- [Hidden Meanings in Plain Sight: RebusBench for Evaluating Cognitive Visual Reasoning](https://mlsanity.com/paper/2604.01764/) (2026)
- [SleepWalk: A Three-Tier Benchmark for Stress-Testing Instruction-Guided Vision-Language Navigation](https://mlsanity.com/paper/2605.10376/) (2026)
- [Do Vision-Language Models Understand 3D Scenes or Just Catalogue Objects?](https://mlsanity.com/paper/2605.20448/) (2026)
- [SCALE: Self-Supervised Constraint-Aware Layout GEneration for Local P&R DRV Fixing at Advanced Nodes](https://mlsanity.com/paper/2607.21850/) (2026)
- [ComplexityWorld: Benchmarking Vision-Language Models on Verifiable Visual Decision Making](https://mlsanity.com/paper/2608.07584/) (2026)
- [PolyComp: A Polycube-based Benchmark for Compositional 3D Spatial Reasoning in Multimodal Models](https://mlsanity.com/paper/2608.14741/) (2026)
- [Investigating Human--AI Discrepancies via Multiple-Solution Problems](https://mlsanity.com/paper/2609.34258/) (2026)

## Abstract (arXiv)

Vision-Language Models (VLMs) are known to struggle with spatial reasoning and visual alignment. To help overcome these limitations, we introduce iVISPAR, an interactive multimodal benchmark designed to evaluate the spatial reasoning capabilities of VLMs acting as agents. mboxiVISPAR is based on a variant of the sliding tile puzzle, a classic problem that demands logical planning, spatial awareness, and multi-step reasoning. The benchmark supports visual 3D, 2D, and text-based input modalities, enabling comprehensive assessments of VLMs' planning and reasoning skills. We evaluate a broad suite of state-of-the-art open-source and closed-source VLMs, comparing their performance while also providing optimal path solutions and a human baseline to assess the task's complexity and feasibility for humans. Results indicate that while VLMs perform better on 2D tasks compared to 3D or text-based settings, they struggle with complex spatial configurations and consistently fall short of human performance, illustrating the persistent challenge of visual alignment. This underscores critical gaps in current VLM capabilities, highlighting their limitations in achieving human-level cognition. Project website: https://microcosm.ai/ivispar