# LOST-3DSG: Lightweight Open-Vocabulary 3D Scene Graphs with Semantic Tracking in Dynamic Environments

arXiv: https://arxiv.org/abs/2601.02905 · HTML: https://mlsanity.com/paper/2601.02905/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 39 references that are in the map.
- First posted on arXiv 6 Jan 2026.

## Builds on (most cited first, 30 of 32)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [OpenScene: 3D Scene Understanding with Open Vocabularies](https://mlsanity.com/paper/2211.15654/) (2022)
- [YOLO-World: Real-Time Open-Vocabulary Object Detection](https://mlsanity.com/paper/2401.17270/) (2024)
- [ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning](https://mlsanity.com/paper/2309.16650/) (2023)
- [Scaling Open-Vocabulary Object Detection](https://mlsanity.com/paper/2306.09683/) (2023)
- [SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planning](https://mlsanity.com/paper/2307.06135/) (2023)
- [3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera](https://mlsanity.com/paper/1910.02527/) (2019)
- [Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructions](https://mlsanity.com/paper/2004.03967/) (2020)
- [Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigation](https://mlsanity.com/paper/2403.17846/) (2024)
- [Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization](https://mlsanity.com/paper/2201.13360/) (2022)
- [SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigation](https://mlsanity.com/paper/2410.08189/) (2024)
- [Semantic Gaussians: Open-Vocabulary Scene Understanding with 3D Gaussian Splatting](https://mlsanity.com/paper/2403.15624/) (2024)
- [Clio: Real-time Task-Driven Open-Set 3D Scene Graphs](https://mlsanity.com/paper/2404.13696/) (2024)
- [Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationships](https://mlsanity.com/paper/2402.12259/) (2024)
- [Open Vocabulary Scene Parsing](https://mlsanity.com/paper/1703.08769/) (2017)
- [Language-Grounded Dynamic Scene Graphs for Interactive Object Search with Mobile Manipulation](https://mlsanity.com/paper/2403.08605/) (2024)
- [OpenNeRF: Open Set 3D Neural Scene Segmentation with Pixel-Wise Features and Rendered Novel Views](https://mlsanity.com/paper/2404.03650/) (2024)
- [RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulation](https://mlsanity.com/paper/2402.15487/) (2024)
- [Dynamic Open-Vocabulary 3D Scene Graphs for Long-term Language-Guided Mobile Manipulation](https://mlsanity.com/paper/2410.11989/) (2024)
- [DELTA: Decomposed Efficient Long-Term Robot Task Planning using Large Language Models](https://mlsanity.com/paper/2404.03275/) (2024)
- [FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interaction](https://mlsanity.com/paper/2503.07909/) (2025)
- [Long-Term Human Trajectory Prediction using 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2405.00552/) (2024)
- [Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks](https://mlsanity.com/paper/1602.00991/) (2016)
- [Beyond MOT: Semantic Multi-Object Tracking](https://mlsanity.com/paper/2403.05021/) (2024)
- [D-Lite: Navigation-Oriented Compression of 3D Scene Graphs for Multi-Robot Collaboration](https://mlsanity.com/paper/2209.06111/) (2022)
- [Social 3D Scene Graphs: Modeling Human Actions and Relations for Interactive Service Robots](https://mlsanity.com/paper/2509.24966/) (2025)
- [Event-Driven Storytelling with Multiple Lifelike Humans in a 3D Scene](https://mlsanity.com/paper/2507.19232/) (2025)
- [EMPOWER: Embodied Multi-role Open-vocabulary Planning with Online Grounding and Execution](https://mlsanity.com/paper/2408.17379/) (2024)

## Built on by (most cited first, 2 of 2)

- [3D Scene Graphs: Open Challenges and Future Directions](https://mlsanity.com/paper/2606.19383/) (2026)
- [A Review Of Robotic World Models For Dynamic Environments Based On Factor And Scene Graphs](https://mlsanity.com/paper/2610.08800/) (2026)

## Abstract (arXiv)

Tracking objects that move within dynamic environments is a core challenge in robotics. Recent research has advanced this topic significantly; however, many existing approaches remain inefficient due to their reliance on heavy foundation models. To address this limitation, we propose LOST-3DSG, a lightweight open-vocabulary 3D scene graph designed to track dynamic objects in real-world environments. Our method adopts a semantic approach to entity tracking based on word2vec and sentence embeddings, enabling an open-vocabulary representation while avoiding the necessity of storing dense CLIP visual features. As a result, LOST-3DSG achieves superior performance compared to approaches that rely on high-dimensional visual embeddings. We evaluate our method through qualitative and quantitative experiments conducted in a real 3D environment using a TIAGo robot. The results demonstrate the effectiveness and efficiency of LOST-3DSG in dynamic object tracking. Code and supplementary material are publicly available on the project website at https://lab-rococo-sapienza.github.io/lost-3dsg/.