mlsanityData to 14 Oct 2026

When Left Is Right? Camera-Motion Understanding beyond Screen

arXiv 2610.17242 ↗First posted 12 Oct 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs also fits: Embodied and 3D vision-language models

Abstract (arXiv)

Can a vision-language model tell left from right? Modern VLMs solve visual puzzles and follow hour-long videos, yet when asked whether the camera moved left or right, most do no better than a coin flip. Adding camera-motion data or geometric representations has been tried, separately, and neither closes the gap. We argue that camera-motion understanding needs two conditions at once: the representation must couple off-screen camera motion with on-screen flow dynamics, and it must be interpreted by the language model rather than trained as a standalone pose or depth estimator. We propose CameraVLM, which pairs a motion-state representation with state-language alignment. Camera tokens capture how the camera moves off screen, flow tokens capture how the subject and background move on screen, and a two-stage training strategy keeps language in the loop so that both are learned together with camera-motion semantics. With only 9B parameters, CameraVLM outperforms all open-source and camera-motion-specific models by a wide margin across three benchmarks and more than 20 baselines, and matches or surpasses GPT-6 Astra as well as human viewers in a 48-participant study. Project page: https://3dagentworld.github.io/CameraVLM/

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 47 of its 64 references that are in the map. First posted on arXiv 12 Oct 2026.

Builds on

47 of 64 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026Geometry-Guided Camera Motion Understanding in VideoLLMs13 Mar
  2. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling21 Jan
  3. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling31 Dec
  4. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May

Other references · 43

  1. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  2. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  3. 2024Grounding Image Matching in 3D with MASt3Rcited by 983
  4. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
  5. 2025Continuous 3D Perception Model with Persistent Statecited by 548
  6. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
  7. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 367
  8. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 287
  9. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 263
  10. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 217
  11. 2025FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Viewscited by 180
  12. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 166
31 more
  1. 2024CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videoscited by 128
  2. 2024BootsTAP: Bootstrapped Training for Tracking-Any-Pointcited by 120
  3. 2025Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memorycited by 87
  4. 2024Local All-Pair Correspondence for Point Trackingcited by 80
  5. 2024DELTA: Dense Efficient Long-range 3D Tracking for any videocited by 65
  6. 2025Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Viewscited by 65
  7. 2025Towards Understanding Camera Motions in Any Videocited by 64
  8. 2023Dense Optical Tracking: Connecting the Dotscited by 61
  9. 2025AllTracker: Efficient Dense Point Tracking at High Resolutioncited by 58
  10. 2025VisualPuzzles: Decoupling Multimodal Reasoning Evaluation from Domain Knowledgecited by 55
  11. 2024TAPTR: Tracking Any Point with Transformers as Detectioncited by 45
  12. 2025VGGT4D: Mining Motion Cues in Visual Geometry Transformers for 4D Scene Reconstructioncited by 28
  13. 2025SpaceMind: Camera-Guided Modality Fusion for Spatial Reasoning in Vision-Language Modelscited by 26
  14. 2024E.T. the Exceptional Trajectories: Text-to-camera-trajectory generation with character awarenesscited by 25
  15. 2025CineTechBench: A Benchmark for Cinematographic Technique Understanding and Generationcited by 23
  16. 2025Outdoor Monocular SLAM with Global Scale-Consistent 3D Gaussian Pointmapscited by 22
  17. 2025RGB-Only Gaussian Splatting SLAM for Unbounded Outdoor Scenescited by 18
  18. 2026CoWTracker: Tracking by Warping instead of Correlationcited by 14
  19. 2025RegGS: Unposed Sparse Views Gaussian Splatting with 3DGS Registrationcited by 13
  20. 2025Tracktention: Leveraging Point Tracking to Attend Videos Faster and Bettercited by 9
  21. 2026CamReasoner: Reinforcing Camera Movement Understanding via Structured Spatial Reasoningcited by 9
  22. 2026HorizonStream: Long-Horizon Attention for Streaming 3D Reconstructioncited by 9
  23. 2025Graph-Guided Scene Reconstruction from Images with 3D Gaussian Splattingcited by 8
  24. 2025Unposed 3DGS Reconstruction with Probabilistic Procrustes Mappingcited by 8
  25. 2026Hy-Embodied-VLM-1.0: Efficient Physical-World Agentscited by 8
  26. 2025LRR-Bench: Left, Right or Rotate? Vision-Language models Still Struggle With Spatial Understanding Taskscited by 6
  27. 2026Unlocking Dense Metric Depth Estimation in VLMscited by 4
  28. 2026TrajTok: Learning Trajectory Tokens enables better Video Understandingcited by 3
  29. 2026Q-GeoMem: Question-Guided Geometric Memory for Video Spatial Reasoningcited by 1
  30. 2026Natural Language Camera Movement Understandingcited by 1
  31. 2026Temporally Grounded Compositional Camera Motion Understanding via Geometric Knowledge Distillationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Lost in Space? Vision-Language Models Struggle with Relative Camera Pose Estimation
  2. 2026Cross-View Action Consistency for Camera-Robust Vision-Language-Action Policies
  3. 2026Rotated, but How Far? Diagnosing and Improving Object-Rotation Reasoning in VLMs
  4. 2025VLM4D: Towards Spatiotemporal Awareness in Vision Language Models
  5. 2026NextMotionQA: Benchmarking and Judging Human Motion Understanding with Vision-Language Models
  6. 2025MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models
  7. 2026MotionVLA: Injecting Geometric Motion into Vision-Language-Action Model
  8. 2026HumanMoveVQA: Can Video MLLMs reason about human movement in videos?