# MIRAGE: A Multi-modal Benchmark for Spatial Perception, Reasoning, and Intelligence

arXiv: https://arxiv.org/abs/2505.10604 · HTML: https://mlsanity.com/paper/2505.10604/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 22 references that are in the map.
- First posted on arXiv 15 May 2025.

## Builds on (most cited first, 16 of 16)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://mlsanity.com/paper/2404.16821/) (2024)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies](https://mlsanity.com/paper/2404.06395/) (2024)
- [Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces](https://mlsanity.com/paper/2412.14171/) (2024)
- [SpatialBot: Precise Spatial Understanding with Vision Language Models](https://mlsanity.com/paper/2406.13642/) (2024)
- [STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?](https://mlsanity.com/paper/2503.23765/) (2025)
- [MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMs](https://mlsanity.com/paper/2503.13111/) (2025)
- [GSR-BENCH: A Benchmark for Grounded Spatial Reasoning Evaluation via Multimodal LLMs](https://mlsanity.com/paper/2406.13246/) (2024)
- [iVISPAR -- An Interactive Visual-Spatial Reasoning Benchmark for VLMs](https://mlsanity.com/paper/2502.03214/) (2025)
- [ST-Think: How Multimodal Large Language Models Reason About 4D Worlds from Ego-Centric Videos](https://mlsanity.com/paper/2503.12542/) (2025)

## Built on by (most cited first, 3 of 3)

- [SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence](https://mlsanity.com/paper/2505.17012/) (2025)
- [Spatial4D-Bench: A Versatile 4D Spatial Intelligence Benchmark](https://mlsanity.com/paper/2601.00092/) (2025)
- [It's Time to Get It Right: Improving Analog Clock Reading and Clock-Hand Spatial Reasoning in Vision-Language Models](https://mlsanity.com/paper/2603.08011/) (2026)

## Abstract (arXiv)

Spatial perception and reasoning are core components of human cognition, encompassing object recognition, spatial relational understanding, and dynamic reasoning. Despite progress in computer vision, existing benchmarks reveal significant gaps in models' abilities to accurately recognize object attributes and reason about spatial relationships, both essential for dynamic reasoning. To address these limitations, we propose MIRAGE, a multi-modal benchmark designed to evaluate models' capabilities in Counting (object attribute recognition), Relation (spatial relational reasoning), and Counting with Relation. Through diverse and complex scenarios requiring fine-grained recognition and reasoning, MIRAGE highlights critical limitations in state-of-the-art models, underscoring the need for improved representations and reasoning frameworks. By targeting these foundational abilities, MIRAGE provides a pathway toward spatiotemporal reasoning in future research.