# Dynamic Open-Vocabulary 3D Scene Graphs for Long-term Language-Guided Mobile Manipulation

arXiv: https://arxiv.org/abs/2410.11989 · HTML: https://mlsanity.com/paper/2410.11989/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2024.

## Built on by (most cited first, 30 of 39)

- [DynaMem: Online Dynamic Spatio-Semantic Memory for Open World Mobile Manipulation](https://mlsanity.com/paper/2411.04999/) (2024)
- [What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Models](https://mlsanity.com/paper/2512.03422/) (2025)
- [KeySG: Hierarchical Keyframe-Based 3D Scene Graphs](https://mlsanity.com/paper/2510.01049/) (2025)
- [MomaGraph: State-Aware Unified Scene Graphs with Vision-Language Model for Embodied Task Planning](https://mlsanity.com/paper/2512.16909/) (2025)
- [DualMap: Online Open-Vocabulary Semantic Mapping for Natural Language Navigation in Dynamic Changing Scenes](https://mlsanity.com/paper/2506.01950/) (2025)
- [Multimodal Spatial Language Maps for Robot Navigation and Manipulation](https://mlsanity.com/paper/2506.06862/) (2025)
- [RAG-3DSG: Enhancing 3D Scene Graphs with Re-Shot Guided Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.10168/) (2026)
- [FunFact: Building Probabilistic Functional 3D Scene Graphs via Factor-Graph Reasoning](https://mlsanity.com/paper/2604.03696/) (2026)
- [3D Scene Graphs: Open Challenges and Future Directions](https://mlsanity.com/paper/2606.19383/) (2026)
- [EmbodiedCoder: Parameterized Embodied Mobile Manipulation via Modern Coding Model](https://mlsanity.com/paper/2510.06207/) (2025)
- [HIMM: Human-Inspired Long-Term Memory Modeling for Embodied Exploration and Question Answering](https://mlsanity.com/paper/2602.15513/) (2026)
- [Can a Robot Walk the Robotic Dog: Triple-Zero Collaborative Navigation for Heterogeneous Multi-Agent Systems](https://mlsanity.com/paper/2603.21723/) (2026)
- [Explore Like Humans: Autonomous Exploration with Online SG-Memo Construction for Embodied Agents](https://mlsanity.com/paper/2604.19034/) (2026)
- [SGN-CIRL: Scene Graph-based Navigation with Curriculum, Imitation, and Reinforcement Learning](https://mlsanity.com/paper/2506.04505/) (2025)
- [R4: Retrieval-Augmented Reasoning for Vision-Language Models in 4D Spatio-Temporal Space](https://mlsanity.com/paper/2512.15940/) (2025)
- [LOST-3DSG: Lightweight Open-Vocabulary 3D Scene Graphs with Semantic Tracking in Dynamic Environments](https://mlsanity.com/paper/2601.02905/) (2026)
- [From Instruction to Event: Sound-Triggered Mobile Manipulation](https://mlsanity.com/paper/2601.21667/) (2026)
- [What if? Emulative Simulation with World Models for Situated Reasoning](https://mlsanity.com/paper/2603.06445/) (2026)
- [SERF: Spatiotemporal Environment and Robot Feature Map for Long-Horizon Mobile Manipulation](https://mlsanity.com/paper/2606.12956/) (2026)
- [Agentic Scene Policies: Unifying Space, Semantics, and Affordances for Robot Action](https://mlsanity.com/paper/2509.19571/) (2025)
- [PIGEON: VLM-Driven Object Navigation via Points of Interest Selection](https://mlsanity.com/paper/2511.13207/) (2025)
- [VLM-DEWM: Dynamic External World Model for Verifiable and Resilient Vision-Language Planning in Manufacturing](https://mlsanity.com/paper/2602.15549/) (2026)
- [EvoMemNav: Efficient Self-Evolving Fine-Grained Memory for Zero-Shot Embodied Navigation](https://mlsanity.com/paper/2606.03509/) (2026)
- [DeWorldSG: Depth-Aware 3D Semantic Scene Graph Generation via World-Model Priors](https://mlsanity.com/paper/2607.00889/) (2026)
- [Just-In-Time Scene Graph Growth: Combating Perceptual Saturation in Long-Horizon Robotics](https://mlsanity.com/paper/2607.13245/) (2026)
- [BINDER: Instantly Adaptive Mobile Manipulation with Open-Vocabulary Commands](https://mlsanity.com/paper/2511.22364/) (2025)
- [LLM-Grounded Dynamic Task Planning with Hierarchical Temporal Logic for Human-Aware Multi-Robot Handover](https://mlsanity.com/paper/2602.09472/) (2026)
- [Knowledge-Guided Manipulation Using Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2603.24083/) (2026)
- [DGSG-Mind: Dynamic 3D Gaussian Scene Graphs for Long-Term Scene Understanding and Grounding](https://mlsanity.com/paper/2605.29879/) (2026)
- [Expanding Spatial and Temporal Context for Robotic Imitation Learning With Scene Graphs](https://mlsanity.com/paper/2606.01072/) (2026)

## Abstract (arXiv)

Enabling mobile robots to perform long-term tasks in dynamic real-world environments is a formidable challenge, especially when the environment changes frequently due to human-robot interactions or the robot's own actions. Traditional methods typically assume static scenes, which limits their applicability in the continuously changing real world. To overcome these limitations, we present DovSG, a novel mobile manipulation framework that leverages dynamic open-vocabulary 3D scene graphs and a language-guided task planning module for long-term task execution. DovSG takes RGB-D sequences as input and utilizes vision-language models (VLMs) for object detection to obtain high-level object semantic features. Based on the segmented objects, a structured 3D scene graph is generated for low-level spatial relationships. Furthermore, an efficient mechanism for locally updating the scene graph, allows the robot to adjust parts of the graph dynamically during interactions without the need for full scene reconstruction. This mechanism is particularly valuable in dynamic environments, enabling the robot to continually adapt to scene changes and effectively support the execution of long-term tasks. We validated our system in real-world environments with varying degrees of manual modifications, demonstrating its effectiveness and superior performance in long-term tasks. Our project page is available at: https://bjhyzj.github.io/dovsg-web.