# Long-Term Human Trajectory Prediction using 3D Dynamic Scene Graphs

arXiv: https://arxiv.org/abs/2405.00552 · HTML: https://mlsanity.com/paper/2405.00552/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 May 2024.

## Built on by (most cited first, 15 of 15)

- [Describe Anything Anywhere At Any Moment](https://mlsanity.com/paper/2512.00565/) (2025)
- [ViLAM: Distilling Vision-Language Reasoning into Attention Maps for Social Robot Navigation](https://mlsanity.com/paper/2503.09820/) (2025)
- [Social 3D Scene Graphs: Modeling Human Actions and Relations for Interactive Service Robots](https://mlsanity.com/paper/2509.24966/) (2025)
- [Long-Term Planning Around Humans in Domestic Environments with 3D Scene Graphs](https://mlsanity.com/paper/2503.09173/) (2025)
- [3D Scene Graphs: Open Challenges and Future Directions](https://mlsanity.com/paper/2606.19383/) (2026)
- [Context-Aware Human Behavior Prediction Using Multimodal Large Language Models: Challenges and Insights](https://mlsanity.com/paper/2504.00839/) (2025)
- [LOST-3DSG: Lightweight Open-Vocabulary 3D Scene Graphs with Semantic Tracking in Dynamic Environments](https://mlsanity.com/paper/2601.02905/) (2026)
- [TH"OR-MAGNI Act: Actions for Human Motion Modeling in Robot-Shared Industrial Spaces](https://mlsanity.com/paper/2412.13729/) (2024)
- [M3DMap: Object-aware Multimodal 3D Mapping for Dynamic Environments](https://mlsanity.com/paper/2508.17044/) (2025)
- [Seeing is Believing (and Predicting): Context-Aware Multi-Human Behavior Prediction with Vision Language Models](https://mlsanity.com/paper/2512.15957/) (2025)
- [HomeEmergency -- Using Audio to Find and Respond to Emergencies in the Home](https://mlsanity.com/paper/2504.01089/) (2025)
- [OpenSGA: Efficient 3D Scene Graph Alignment in the Open World](https://mlsanity.com/paper/2605.10484/) (2026)
- [EgoTrack3D: A Modular Framework for Egocentric 3D Object Tracking](https://mlsanity.com/paper/2608.08016/) (2026)
- [HINT-Plan: Human Intention-Aware Robot Task Planning in Context-Rich Environments using Vision Language Models](https://mlsanity.com/paper/2609.17771/) (2026)
- [A Review Of Robotic World Models For Dynamic Environments Based On Factor And Scene Graphs](https://mlsanity.com/paper/2610.08800/) (2026)

## Abstract (arXiv)

We present a novel approach for long-term human trajectory prediction in indoor human-centric environments, which is essential for long-horizon robot planning in these environments. State-of-the-art human trajectory prediction methods are limited by their focus on collision avoidance and short-term planning, and their inability to model complex interactions of humans with the environment. In contrast, our approach overcomes these limitations by predicting sequences of human interactions with the environment and using this information to guide trajectory predictions over a horizon of up to 60s. We leverage Large Language Models (LLMs) to predict interactions with the environment by conditioning the LLM prediction on rich contextual information about the scene. This information is given as a 3D Dynamic Scene Graph that encodes the geometry, semantics, and traversability of the environment into a hierarchical representation. We then ground these interaction sequences into multi-modal spatio-temporal distributions over human positions using a probabilistic approach based on continuous-time Markov Chains. To evaluate our approach, we introduce a new semi-synthetic dataset of long-term human trajectories in complex indoor environments, which also includes annotations of human-object interactions. We show in thorough experimental evaluations that our approach achieves a 54% lower average negative log-likelihood and a 26.5% lower Best-of-20 displacement error compared to the best non-privileged (i.e., evaluated in a zero-shot fashion on the dataset) baselines for a time horizon of 60s.