mlsanityData to 7 Oct 2026

CaMo: Camera Motion Grounded Evaluation and Training for Vision-Language Models

arXiv 2605.20165 ↗First posted 19 May 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Vision-Language Models (VLMs) achieve strong performance on spatial question answering benchmarks, yet it remains unclear whether such gains reflect genuine spatial intelligence. We show that existing spatial VLMs lack basic camera motion understanding, a key component of spatial cognition. We propose the Spatial Narrative Score (SNS), an evaluation framework that requires VLMs to generate explicit spatial narratives capturing both scene semantics and camera motion, followed by reasoning with a frozen proxy LLM. Under SNS, state-of-the-art spatial VLMs exhibit significant performance degradation despite high direct question answering accuracy. To address this gap, we introduce CaMo, a camera motion grounded VLM that achieves consistent performance across SNS evaluation and direct spatial question answering accuracy. Our results highlight the importance of explicit spatial narrative externalization for evaluating VLMs with transferable 3D spatial understanding. Our code, data, and model is available at https://github.com/hsiangwei0903/CaMo

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 49 of its 52 references that are in the map. First posted on arXiv 19 May 2026.

Builds on

49 of 52 references.

Closest ancestors · 12

Same direction, newest first

  1. 2026Can LLMs See Without Pixels? Benchmarking Spatial Intelligence from Textual Descriptions7 Jan
  2. 2025SpaceMind: Camera-Guided Modality Fusion for Spatial Reasoning in Vision-Language Models28 Nov
  3. 2025SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Models9 Oct
  4. 20253D-R1: Enhancing Reasoning in 3D VLMs for Unified Scene Understanding31 Jul
  5. 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing11 Jun
  6. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence29 May
  7. 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models27 May
  8. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction26 May
  9. 2025SpaceR: Reinforcing MLLMs in Video Spatial Reasoning2 Apr
  10. 2025Improved Visual-Spatial Reasoning via R1-Zero-Like Training1 Apr
  11. 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D29 Mar
  12. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec

Other references · 37

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2025Qwen2.5-VL Technical Reportcited by 5,418
  5. 2024GPT-4o System Cardcited by 4,467
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  7. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  8. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  9. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  10. 2024Mixtral of Expertscited by 1,785
  11. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  12. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
25 more
  1. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  2. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  3. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  4. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  5. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  6. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
  7. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  8. 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 464
  9. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  10. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
  11. 2025Kimi-VL Technical Reportcited by 381
  12. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  13. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 310
  14. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 285
  15. 2024Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motioncited by 167
  16. 2022Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wildcited by 159
  17. 2024Tarsier: Recipes for Training and Evaluating Large Video Description Modelscited by 140
  18. 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 137
  19. 2024MotionBooth: Motion-Aware Customized Text-to-Video Generationcited by 92
  20. 2025Describe Anything: Detailed Localized Image and Video Captioningcited by 84
  21. 2025Towards Understanding Camera Motions in Any Videocited by 62
  22. 2024Learning Visual Grounding from Generative Vision and Language Modelcited by 27
  23. 2025Benchmark Designers Should "Train on the Test Set" to Expose Exploitable Non-Visual Shortcutscited by 26
  24. 2025Scene-R1: Video-Grounded Large Language Models for 3D Scene Reasoning without 3D Annotationscited by 14
  25. 2025VDC-Agent: When Video Detailed Captioners Evolve Themselves via Agentic Self-Reflectioncited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models
  2. 2025SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data
  3. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
  4. 2026SpatialChain: A Benchmark for Auditing Spatial Reasoning Faithfulness in VLMs
  5. 2026MindEdit-Bench: Benchmarking Object-Level Counterfactual Spatial Reasoning in VLMs from In-the-Wild Photos
  6. 2026SpatiaLab: Can Vision-Language Models Perform Spatial Reasoning in the Wild?
  7. 2024Reasoning Paths with Reference Objects Elicit Quantitative Spatial Reasoning in Large Vision-Language Models
  8. 2025Spatial-DISE: A Unified Benchmark for Evaluating Spatial Reasoning in Vision-Language Models