mlsanityData to 14 Oct 2026

3D-R1: Enhancing Reasoning in 3D VLMs for Unified Scene Understanding

arXiv 2507.23478 ↗First posted 31 Jul 2025cs.CVcited by 47

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more
Abstract (arXiv)

Large vision-language models (VLMs) have made significant strides in 2D visual understanding tasks, sparking interest in extending these capabilities to 3D scene understanding. However, current 3D VLMs often struggle with robust reasoning and generalization due to limitations in high-quality spatial data and the static nature of viewpoint assumptions. To address these challenges, we propose 3D-R1, a foundation model that enhances the reasoning capabilities of 3D VLMs. Specifically, we first construct a high-quality synthetic dataset with CoT, named Scene-30K, leveraging existing 3D-VL datasets and a data engine based on Gemini 2.5 Pro. It serves as cold-start initialization data for 3D-R1. Moreover, we leverage RLHF policy such as GRPO in the reinforcement learning training process to enhance reasoning capabilities and introduce three reward functions: a perception reward, a semantic similarity reward and a format reward to maintain detection accuracy and answer semantic precision. Furthermore, we introduce a dynamic view selection strategy that adaptively chooses the most informative perspectives for 3D scene understanding. Extensive experiments demonstrate that 3D-R1 delivers an average improvement of 10% across various 3D scene benchmarks, highlighting its effectiveness in enhancing reasoning and generalization in 3D scene understanding. Code: https://github.com/AIGeeksGroup/3D-R1. Website: https://aigeeksgroup.github.io/3D-R1.

Key facts. Cited by 47 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 19 of its 71 references that are in the map. First posted on arXiv 31 Jul 2025.

Builds on

19 of 71 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025GaussianVLM: Scene-centric 3D Vision-Language Models using Language-aligned Gaussian Splats for Embodied Reasoning and Beyond1 Jul
  2. 2025Scene-R1: Video-Grounded Large Language Models for 3D Scene Reasoning without 3D Annotations21 Jun

Other references · 17

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2024Depth Anything V2cited by 1,422
  5. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  6. 20223D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selectioncited by 110
  7. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Modelscited by 109
  8. 2021D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Groundingcited by 52
  9. 2024LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferencescited by 38
  10. 2022Contextual Modeling for 3D Dense Captioning on Point Cloudscited by 18
  11. 2025DC-Scene: Data-Centric Learning for 3D Scene Understandingcited by 15
  12. 20253D CoCa: Contrastive Learners are 3D Captionerscited by 13
5 more
  1. 2025Compile Scene Graphs with Reinforcement Learningcited by 11
  2. 2025DIP-R1: Deep Inspection and Perception with RL Looking Through and Understanding Complex Scenescited by 11
  3. 2025Evaluating Gemini in an arena for learningcited by 11
  4. 2025Spatial 3D-LLM: Exploring Spatial Awareness in 3D Vision-Language Modelscited by 8
  5. 2024OpenScan: A Benchmark for Generalized Open-Vocabulary 3D Scene Understandingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 14; 2026: 33.

47 papers.

Sort:

By area:

  1. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  2. 2025VLA-R1: Enhancing Reasoning in Vision-Language-Action Modelscited by 38
  3. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecyclecited by 35
  4. 2025MobileVLA-R1: Reinforcing Vision-Language-Action for Mobile Robotscited by 15
  5. 2026ReVSI: Rebuilding Visual Spatial Intelligence Evaluation for Accurate Assessment of VLM 3D Reasoningcited by 15
  6. 2026Thinking in Dynamics: How Multimodal Large Language Models Perceive, Track, and Reason Dynamics in Physical 4D Worldcited by 12
  7. 2025EvoVLA: Self-Evolving Vision-Language-Action Modelcited by 11
  8. 2025StereoAdapter: Adapting Stereo Depth Estimation to Underwater Scenescited by 9
  9. 2026PanoEnv: Exploring 3D Spatial Intelligence in Panoramic Environments with Reinforcement Learningcited by 7
  10. 2025VaseVQA: Multimodal Agent and Benchmark for Ancient Greek Potterycited by 6
  11. 2026GeoWorld: Geometric World Modelscited by 6
  12. 2025EgoLCD: Egocentric Video Generation with Long Context Diffusioncited by 5
35 more
  1. 2026AnyDepth: Depth Estimation Made Easycited by 5
  2. 2025VaseVQA-3D: Benchmarking 3D VLMs on Ancient Greek Potterycited by 4
  3. 2025Video2Layout: Recall and Reconstruct Metric-Grounded Cognitive Map for Spatial Reasoningcited by 4
  4. 20263D-Layout-R1: Structured Reasoning for Language-Instructed Spatial Editingcited by 4
  5. 2025PanoGrounder: Bridging 2D and 3D with Panoramic Scene Representations for VLM-based 3D Visual Groundingcited by 3
  6. 20263D CoCa v2: Contrastive Learners with Test-Time Search for Generalizable Spatial Intelligencecited by 3
  7. 2026VIEW2SPACE: Studying Multi-View Visual Reasoning from Sparse Observationscited by 3
  8. 2026Grounded 3D-Aware Spatial Vision-Language Modelingcited by 3
  9. 2026DriveJudge: Rethinking Autonomous Driving Evaluation with Vision-Language Modelscited by 3
  10. 2026Dense Reward for Multi-View 3D Reasoning with Global Maps and Local Viewscited by 3
  11. 2025Multimodal Data Storage and Retrieval for Embodied AI: A Surveycited by 2
  12. 2026Smooth Operator: Smooth Verifiable Reward Activates Spatial Reasoning Ability of Vision-Language Modelcited by 2
  13. 2026OnlineSI: Taming Large Language Model for Online 3D Understanding and Groundingcited by 2
  14. 2026UniMesh: Unifying 3D Mesh Understanding and Generationcited by 2
  15. 2026LMM-Track4D: Eliciting 4D Dynamic Reasoning in LMMs via Trajectory-Grounded Dialoguecited by 2
  16. 2026Distilling Neuro-Symbolic Programs into 3D Multi-modal LLMscited by 2
  17. 2025Error-Driven Scene Editing for 3D Grounding in Large Language Modelscited by 1
  18. 20263D-RFT: Reinforcement Fine-Tuning for Video-based 3D Scene Understandingcited by 1
  19. 2026UniGround: Universal 3D Visual Grounding via Training-Free Scene Parsingcited by 1
  20. 2026Seeing Once is Enough? Online Geometry-Aware Token Pruning for 3D Question Answeringcited by 1
  21. 2026ConsiSpace: Learning Geometric Consistency Matters for Video Spatial Reasoningcited by 1
  22. 2026MobileVLA-R1 2.0: RL-Enhanced Reasoning for Mobile Robot Controlcited by 1
  23. 2025OpenGround: Planning-based Online Perception for Open-World 3D Visual Groundingcited by 0
  24. 2026Token Warping Helps MLLMs Look from Nearby Viewpointscited by 0
  25. 2026Last-Layer-Centric Feature Recombination: Unleashing 3D Geometric Knowledge in DINOv3 for Monocular Depth Estimationcited by 0
  26. 2026CaMo: Camera Motion Grounded Evaluation and Training for Vision-Language Modelscited by 0
  27. 2026Holo-Captioning: Toward the Text Equivalent of 3D Scenescited by 0
  28. 2026VaseMuseum: Digital Intelligent Museum for Ancient Greek Potterycited by 0
  29. 2026CoordRefer: Coordinate-Aware 3D Visual Grounding from Multiview Imagescited by 0
  30. 2026RefineAny3D: Depth Refinement as Semantic Alignment for Monocular 3D Detectioncited by 0
  31. 2026SCOUT: Unlocking Enhanced Spatial Reasoning via Structured Chain-of-Thought and Multi-Objective Process Rewardcited by 0
  32. 2026CoVeR: Coverage-Based Token Pruning for Multi-View 3D Reasoning in VLMscited by 0
  33. 2026Metric-Bench: Exploring In-context Spatial Metric Reasoning in VLMs for Indoor Scenescited by 0
  34. 2026From Reasoning Failures to Composable Video Spatial Intelligencecited by 0
  35. 2026WOVEN: Weaving Visual World Modeling into Multimodal LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D
  2. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction
  3. 2025G2VLM: Geometry Grounded Vision Language Model with Unified 3D Reconstruction and Spatial Reasoning
  4. 20253DRS: MLLMs Need 3D-Aware Representation Supervision for Scene Understanding
  5. 20253D-Aware Vision-Language Models Fine-Tuning with Geometric Distillation
  6. 2026Enhancing MLLM Spatial Understanding via Active 3D Scene Exploration for Multi-Perspective Reasoning
  7. 2026Reinforcing Dual-Path Reasoning in Spatial Vision Language Models
  8. 2025LEO-VL: Efficient Scene Representation for Scalable 3D Vision-Language Learning