mlsanityData to 7 Oct 2026

CoSpace: Benchmarking Continuous Space Perception Ability for Vision-Language Models

arXiv 2503.14161 ↗First posted 18 Mar 2025cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Vision-Language Models (VLMs) have recently witnessed significant progress in visual comprehension. As the permitting length of image context grows, VLMs can now comprehend a broader range of views and spaces. Current benchmarks provide insightful analysis of VLMs in tasks involving complex visual instructions following, multi-image understanding and spatial reasoning. However, they usually focus on spatially irrelevant images or discrete images captured from varied viewpoints. The compositional characteristic of images captured from a static viewpoint remains underestimated. We term this characteristic as Continuous Space Perception. When observing a scene from a static viewpoint while shifting orientations, it produces a series of spatially continuous images, enabling the reconstruction of the entire space. In this paper, we present CoSpace, a multi-image visual understanding benchmark designed to assess the Continuous Space perception ability for VLMs. CoSpace contains 2,918 images and 1,626 question-answer pairs, covering seven types of tasks. We conduct evaluation across 19 proprietary and open-source VLMs. Results reveal that there exist pitfalls on the continuous space perception ability for most of the evaluated models, including proprietary ones. Interestingly, we find that the main discrepancy between open-source and proprietary models lies not in accuracy but in the consistency of responses. We believe that enhancing the ability of continuous space perception is essential for VLMs to perform effectively in real-world tasks and encourage further research to advance this capability.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 50 of its 61 references that are in the map. First posted on arXiv 18 Mar 2025.

Builds on

50 of 61 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024An Empirical Analysis on Spatial Reasoning Capabilities of Large Multimodal Models9 Nov
  2. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Models17 Jul
  3. 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models21 Jun
  4. 2024GSR-BENCH: A Benchmark for Grounded Spatial Reasoning Evaluation via Multimodal LLMs19 Jun
  5. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan
  6. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation24 Dec
  7. 2023Towards End-to-End Embodied Decision Making via Multi-modal Large Language Model: Explorations with GPT4-Vision and Beyond3 Oct

Other references · 43

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  2. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  3. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  4. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  5. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  6. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  7. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  8. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  9. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  10. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  11. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  12. 2023VILA: On Pre-training for Visual Language Modelscited by 686
31 more
  1. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  2. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
  3. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
  4. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  5. 2024Long Context Transfer from Language to Visioncited by 502
  6. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  7. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 285
  8. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 284
  9. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 283
  10. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  11. 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 212
  12. 2023Video Understanding with Large Language Models: A Surveycited by 203
  13. 2023MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learningcited by 171
  14. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 130
  15. 2024LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via a Hybrid Architecturecited by 100
  16. 2024Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequencescited by 93
  17. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 93
  18. 2024MileBench: Benchmarking MLLMs in Long Contextcited by 86
  19. 2024Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-trainingcited by 82
  20. 2023Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructionscited by 76
  21. 2024Explore until Confident: Efficient Exploration for Embodied Question Answeringcited by 74
  22. 2024Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphscited by 72
  23. 2023SayNav: Grounding Large Language Models for Dynamic Planning to Navigation in New Environmentscited by 53
  24. 2024MIBench: Evaluating Multimodal Large Language Models over Multiple Imagescited by 36
  25. 2023Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Modelscited by 28
  26. 2024Towards Multimodal In-Context Learning for Vision & Language Modelscited by 24
  27. 2024Leopard: A Vision Language Model For Text-Rich Multi-Image Taskscited by 18
  28. 2024ReALFRED: An Embodied Instruction Following Benchmark in Photo-Realistic Environmentscited by 16
  29. 2024CODIS: Benchmarking Context-Dependent Visual Comprehension for Multimodal Large Language Modelscited by 11
  30. 2024ActiView: Evaluating Active Perception Ability for Multimodal Large Language Modelscited by 11
  31. 2024Browse and Concentrate: Comprehending Multimodal Content via prior-LLM Context Fusioncited by 5

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Thinking with Visual Abstract: Enhancing Multimodal Reasoning via Visual Abstractioncited by 15

Similar, not cited

Close in wording, no citation either way

  1. 2025LAST: LeArning to Think in Space and Time for Generalist Vision-Language Models
  2. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
  3. 2025VLM4D: Towards Spatiotemporal Awareness in Vision Language Models
  4. 2025Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens
  5. 2026The Last Visible Pixel: Probing Fine-Scale Perception in Vision-Language Models
  6. 2026ViSTR-Bench: Can MLLMs Reason from Continuous Visual Cues in Dynamic Scenes?
  7. 2025CIVET: Systematic Evaluation of Understanding in VLMs
  8. 2025SD-VLM: Spatial Measuring and Understanding with Depth-Encoded Vision-Language Models