mlsanityData to 14 Oct 2026

D3D-VLP: Dynamic 3D Vision-Language-Planning Model for Embodied Grounding and Navigation

arXiv 2512.12622 ↗First posted 14 Dec 2025cs.CV cs.ROcited by 5

Not on the map. Why?

Abstract (arXiv)

Embodied agents face a critical dilemma that end-to-end models lack interpretability and explicit 3D reasoning, while modular systems ignore cross-component interdependencies and synergies. To bridge this gap, we propose the Dynamic 3D Vision-Language-Planning Model (D3D-VLP). Our model introduces two key innovations: 1) A Dynamic 3D Chain-of-Thought (3D CoT) that unifies planning, grounding, navigation, and question answering within a single 3D-VLM and CoT pipeline; 2) A Synergistic Learning from Fragmented Supervision (SLFS) strategy, which uses a masked autoregressive loss to learn from massive and partially-annotated hybrid data. This allows different CoT components to mutually reinforce and implicitly supervise each other. To this end, we construct a large-scale dataset with 10M hybrid samples from 5K real scans and 20K synthetic scenes that are compatible with online learning methods such as RL and DAgger. Our D3D-VLP achieves state-of-the-art results on multiple benchmarks, including Vision-and-Language Navigation (R2R-CE, REVERIE-CE, NavRAG-CE), Object-goal Navigation (HM3D-OVON), and Task-oriented Sequential Grounding and Navigation (SG3D). Real-world mobile manipulation experiments further validate the effectiveness.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 71 of its 78 references that are in the map. First posted on arXiv 14 Dec 2025.

Builds on

71 of 78 references.

References · 71

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  3. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  4. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  5. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  6. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  7. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  8. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 454
  9. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
  10. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  11. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  12. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 380
38 more
  1. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 365
  2. 2023An Embodied Generalist Agent in 3D Worldcited by 365
  3. 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 346
  4. 2023Scaling Open-Vocabulary Object Detectioncited by 343
  5. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 333
  6. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
  7. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 211
  8. 2024NVILA: Efficient Frontier Visual Language Modelscited by 209
  9. 2022AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domainscited by 208
  10. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 205
  11. 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 204
  12. 2019RIO: 3D Object Instance Re-Localization in Changing Indoor Environmentscited by 188
  13. 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awarenesscited by 183
  14. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 169
  15. 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 167
  16. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 157
  17. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 156
  18. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 146
  19. 2022Multi-View Transformer for 3D Visual Groundingcited by 140
  20. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 138
  21. 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understandingcited by 138
  22. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 133
  23. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 132
  24. 2022Language Conditioned Spatial Relation Reasoning for 3D Object Groundingcited by 122
  25. 2025Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priorscited by 120
  26. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 118
  27. 2024Grounded 3D-LLM with Referent Tokenscited by 115
  28. 2022Habitat-Matterport 3D Semantics Datasetcited by 110
  29. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 98
  30. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 94
  31. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  32. 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
  33. 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 79
  34. 2025Embodied Navigation Foundation Modelcited by 72
  35. 2024HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigationcited by 71
  36. 2024Unifying 3D Vision-Language Understanding via Promptable Queriescited by 70
  37. 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 67
  38. 2024Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigationcited by 65

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026EmbodiedSplat: Online Feed-Forward Semantic 3DGS for Open-Vocabulary 3D Scene Understandingcited by 1
  2. 2026What Limits Vision-and-Language Navigation ?cited by 1
  3. 2026Image2Sim: Scaling Embodied Navigation via Generative Neural Simulatorcited by 1
  4. 2026NoPA: Non-Parametric Online 3D Scene Graph Generationcited by 0
  5. 2026From Region Arrival to Instance-Level Grounding in Vision-and-Language Navigationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Grounding Vision and Language to 3D Masks for Long-Horizon Box Rearrangement
  2. 2025Abstract 3D Perception for Spatial Intelligence in Vision-Language Models
  3. 20243D-VLA: A 3D Vision-Language-Action Generative World Model
  4. 2026SoftNav: Injecting 3D Scene Tokens into VLMs for Embodied Navigation
  5. 20253D-R1: Enhancing Reasoning in 3D VLMs for Unified Scene Understanding
  6. 2026Traj-VLN: Learning Pixel-Space Interaction via Autoregressive Trajectory Generation
  7. 2026AgentVLN: Towards Agentic Vision-and-Language Navigation
  8. 2024VLA-3D: A Dataset for 3D Semantic Scene Understanding and Navigation