# Long-Term Planning Around Humans in Domestic Environments with 3D Scene Graphs

arXiv: https://arxiv.org/abs/2503.09173 · HTML: https://mlsanity.com/paper/2503.09173/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 12 Mar 2025.

## Builds on (most cited first, 20 of 20)

- [LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action](https://mlsanity.com/paper/2207.04429/) (2022)
- [3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera](https://mlsanity.com/paper/1910.02527/) (2019)
- [Human Motion Trajectory Prediction: A Survey](https://mlsanity.com/paper/1905.06113/) (2019)
- [Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructions](https://mlsanity.com/paper/2004.03967/) (2020)
- [Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigation](https://mlsanity.com/paper/2403.17846/) (2024)
- [3D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans](https://mlsanity.com/paper/2002.06289/) (2020)
- [Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2101.06894/) (2021)
- [Reward-rational (implicit) choice: A unifying formalism for reward learning](https://mlsanity.com/paper/2002.04833/) (2020)
- [SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigation](https://mlsanity.com/paper/2410.08189/) (2024)
- [3-D Scene Graph: A Sparse and Semantic Representation of Physical Environments for Intelligent Agents](https://mlsanity.com/paper/1908.04929/) (2019)
- [Language-Grounded Dynamic Scene Graphs for Interactive Object Search with Mobile Manipulation](https://mlsanity.com/paper/2403.08605/) (2024)
- [Continual Learning for Real-World Autonomous Systems: Algorithms, Challenges and Frameworks](https://mlsanity.com/paper/2105.12374/) (2021)
- [Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks](https://mlsanity.com/paper/2108.01176/) (2021)
- [CoNVOI: Context-aware Navigation using Vision Language Models in Outdoor and Indoor Environments](https://mlsanity.com/paper/2403.15637/) (2024)
- [Collaborative Dynamic 3D Scene Graphs for Automated Driving](https://mlsanity.com/paper/2309.06635/) (2023)
- [DELTA: Decomposed Efficient Long-Term Robot Task Planning using Large Language Models](https://mlsanity.com/paper/2404.03275/) (2024)
- [Optimal Scene Graph Planning with Large Language Model Guidance](https://mlsanity.com/paper/2309.09182/) (2023)
- [Long-Term Human Trajectory Prediction using 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2405.00552/) (2024)
- [PREDILECT: Preferences Delineated with Zero-Shot Language-based Reasoning in Reinforcement Learning](https://mlsanity.com/paper/2402.15420/) (2024)
- [CoMet: Modeling Group Cohesion for Socially Compliant Robot Navigation in Crowded Scenes](https://mlsanity.com/paper/2108.09848/) (2021)

## Built on by (most cited first, 4 of 4)

- [KeySG: Hierarchical Keyframe-Based 3D Scene Graphs](https://mlsanity.com/paper/2510.01049/) (2025)
- [Social 3D Scene Graphs: Modeling Human Actions and Relations for Interactive Service Robots](https://mlsanity.com/paper/2509.24966/) (2025)
- [3D Scene Graphs: Open Challenges and Future Directions](https://mlsanity.com/paper/2606.19383/) (2026)
- [GESTO: Human-Centric Spatio-Temporal Memory for Reasoning in Dynamic Scenes](https://mlsanity.com/paper/2608.10886/) (2026)

## Abstract (arXiv)

Long-term planning for robots operating in domestic environments poses unique challenges due to the interactions between humans, objects, and spaces. Recent advancements in trajectory planning have leveraged vision-language models (VLMs) to extract contextual information for robots operating in real-world environments. While these methods achieve satisfying performance, they do not explicitly model human activities. Such activities influence surrounding objects and reshape spatial constraints. This paper presents a novel approach to trajectory planning that integrates human preferences, activities, and spatial context through an enriched 3D scene graph (3DSG) representation. By incorporating activity-based relationships, our method captures the spatial impact of human actions, leading to more context-sensitive trajectory adaptation. Preliminary results demonstrate that our approach effectively assigns costs to spaces influenced by human activities, ensuring that the robot trajectory remains contextually appropriate and sensitive to the ongoing environment. This balance between task efficiency and social appropriateness enhances context-aware human-robot interactions in domestic settings. Future work includes implementing a full planning pipeline and conducting user studies to evaluate trajectory acceptability.