# Video Spatial Reasoning with Object-Centric 3D Rollout

arXiv: https://arxiv.org/abs/2511.13190 · HTML: https://mlsanity.com/paper/2511.13190/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 30 of its 41 references that are in the map.
- First posted on arXiv 17 Nov 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)
- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 30 of 30)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [LLaVA-Video: Video Instruction Tuning With Synthetic Data](https://mlsanity.com/paper/2410.02713/) (2024)
- [Video-R1: Reinforcing Video Reasoning in MLLMs](https://mlsanity.com/paper/2503.21776/) (2025)
- [ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data](https://mlsanity.com/paper/2111.08897/) (2021)
- [DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning](https://mlsanity.com/paper/2505.14362/) (2025)
- [Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence](https://mlsanity.com/paper/2505.23747/) (2025)
- [VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction](https://mlsanity.com/paper/2505.20279/) (2025)
- [SpaceR: Reinforcing MLLMs in Video Spatial Reasoning](https://mlsanity.com/paper/2504.01805/) (2025)
- [SpatialBot: Precise Spatial Understanding with Vision Language Models](https://mlsanity.com/paper/2406.13642/) (2024)
- [From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D](https://mlsanity.com/paper/2503.22976/) (2025)
- [Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priors](https://mlsanity.com/paper/2505.24625/) (2025)
- [Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs](https://mlsanity.com/paper/2505.15436/) (2025)
- [NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation](https://mlsanity.com/paper/2504.13055/) (2025)
- [Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awareness](https://mlsanity.com/paper/2504.01901/) (2025)
- [On Pursuit of Designing Multi-modal Transformer for Video Grounding](https://mlsanity.com/paper/2109.06085/) (2021)
- [Improved Visual-Spatial Reasoning via R1-Zero-Like Training](https://mlsanity.com/paper/2504.00883/) (2025)
- [Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2505.20272/) (2025)
- [Continual LLaVA: Continual Instruction Tuning in Large Vision-Language Models](https://mlsanity.com/paper/2411.02564/) (2024)
- [PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videos](https://mlsanity.com/paper/2412.01800/) (2024)
- [PPLLaVA: Varied Video Sequence Understanding With Prompt Guidance](https://mlsanity.com/paper/2411.02327/) (2024)
- [RAP: Efficient Text-Video Retrieval with Sparse-and-Correlated Adapter](https://mlsanity.com/paper/2405.19465/) (2024)
- [Video SimpleQA: Towards Factuality Evaluation in Large Video Language Models](https://mlsanity.com/paper/2503.18923/) (2025)

## Built on by (most cited first, 6 of 6)

- [Think3D: Thinking with Space for Spatial Reasoning](https://mlsanity.com/paper/2601.13029/) (2026)
- [ReVSI: Rebuilding Visual Spatial Intelligence Evaluation for Accurate Assessment of VLM 3D Reasoning](https://mlsanity.com/paper/2604.24300/) (2026)
- [RieMind: Geometry-Grounded Spatial Agent for Scene Understanding](https://mlsanity.com/paper/2603.15386/) (2026)
- [ZeroSplat: Generalized Referring Segmentation in 3D Gaussian Splatting](https://mlsanity.com/paper/2607.18801/) (2026)
- [VideoKR: Towards Knowledge- and Reasoning-Intensive Video Understanding](https://mlsanity.com/paper/2606.05259/) (2026)
- [Skill-3D: Evolving Scene-Aware Skills for Agentic 3D Spatial Reasoning](https://mlsanity.com/paper/2606.07436/) (2026)

## Abstract (arXiv)

Recent advances in Multi-modal Large Language Models (MLLMs) have showcased remarkable capabilities in vision-language understanding. However, enabling robust video spatial reasoning-the ability to comprehend object locations, orientations, and inter-object relationships in dynamic 3D scenes-remains a key unsolved challenge. Existing approaches primarily rely on spatially grounded supervised fine-tuning or reinforcement learning, yet we observe that such models often exhibit query-locked reasoning, focusing narrowly on objects explicitly mentioned in the prompt while ignoring critical contextual cues. To address this limitation, we propose Object-Centric 3D Rollout (OCR), a novel strategy that introduces structured perturbations to the 3D geometry of selected objects during training. By degrading object-specific visual cues and projecting the altered geometry into 2D space, OCR compels the model to reason holistically across the entire scene. We further design a rollout-based training pipeline that jointly leverages vanilla and region-noisy videos to optimize spatial reasoning trajectories. Experiments demonstrate state-of-the-art performance: our 3B-parameter model achieves 47.5% accuracy on VSI-Bench, outperforming several 7B baselines. Ablations confirm OCR's superiority over prior rollout strategies (e.g., T-GRPO, NoisyRollout).