mlsanityData to 14 Oct 2026

Visuospatial Cognitive Assistant

arXiv 2505.12312 ↗First posted 18 May 2025cs.CV cs.AI cs.CLcited by 11

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Video-based spatial cognition is vital for robotics and embodied AI but challenges current Vision-Language Models (VLMs). This paper makes two key contributions. First, we introduce ViCA (Visuospatial Cognitive Assistant)-322K, a diverse dataset of 322,003 QA pairs from real-world indoor videos (ARKitScenes, ScanNet, ScanNet++), offering supervision for 3D metadata-grounded queries and video-based complex reasoning. Second, we develop ViCA-7B, fine-tuned on ViCA-322K, which achieves new state-of-the-art on all eight VSI-Bench tasks, outperforming existing models, including larger ones (e.g., +26.1 on Absolute Distance). For interpretability, we present ViCA-Thinking-2.68K, a dataset with explicit reasoning chains, and fine-tune ViCA-7B to create ViCA-7B-Thinking, a model that articulates its spatial reasoning. Our work highlights the importance of targeted data and suggests paths for improved temporal-spatial modeling. We release all resources to foster research in robust visuospatial intelligence.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 16 of its 40 references that are in the map. First posted on arXiv 18 May 2025.

Builds on

16 of 40 references.

References · 16

  1. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  5. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  6. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  7. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  8. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  9. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  10. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
  11. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  12. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 454
4 more
  1. 2024Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Datacited by 67
  2. 2023GlueStick: Robust Image Matching by Sticking Points and Lines Togethercited by 66
  3. 2024OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Textcited by 59
  4. 2024MINT-1T: Scaling Open-Source Multimodal Data by 10x: A Multimodal Dataset with One Trillion Tokenscited by 50

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 8.

11 papers.

  1. 2025SpaceMind: Camera-Guided Modality Fusion for Spatial Reasoning in Vision-Language Modelscited by 26
  2. 2025Beyond Flatlands: Unlocking Spatial Intelligence by Decoupling 3D Reasoning from Numerical Regressioncited by 8
  3. 2026GEM: Generative Supervision Helps Embodied Intelligencecited by 6
  4. 2025Video2Layout: Recall and Reconstruct Metric-Grounded Cognitive Map for Spatial Reasoningcited by 4
  5. 2026Towards Predictive, Aligned, and Scalable Robot Learningcited by 3
  6. 2026Chatting with Images for Introspective Visual Thinkingcited by 1
  7. 2026Q-GeoMem: Question-Guided Geometric Memory for Video Spatial Reasoningcited by 1
  8. 2026Ouroboros-Spatial: Closing the Data-Model Loop for Spatial Reasoningcited by 1
  9. 2026OneCanvas: 3D Scene Understanding via Panoramic Reprojectioncited by 0
  10. 2026SpaR3D-MoE: Adaptive 3D Spatial Reasoning from Sparse Views Meets Geometry-Inductive Mixture-of-Expertscited by 0
  11. 2026Chain of Spatial Thoughts: Modality-Agnostic Spatial Grounding for Vision Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Visuospatial Cognition via Hierarchical Fusion of Visual Experts
  2. 2025InternSpatial: A Comprehensive Dataset for Spatial Reasoning in Vision-Language Models
  3. 2026ViSRA: A Video-based Spatial Reasoning Agent for Multi-modal Large Language Models
  4. 2026ESPIRE: A Diagnostic Benchmark for Embodied Spatial Reasoning of Vision-Language Models
  5. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective
  6. 2026ViSTR-Bench: Can MLLMs Reason from Continuous Visual Cues in Dynamic Scenes?
  7. 2025Improved Visual-Spatial Reasoning via R1-Zero-Like Training
  8. 2026MosaicThinker: On-Device Visual Spatial Reasoning for Embodied AI via Iterative Construction of Space Representation