# Translating Universal Scene Descriptions into Knowledge Graphs for Robotic Environment

arXiv: https://arxiv.org/abs/2310.16737 · HTML: https://mlsanity.com/paper/2310.16737/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 23 references that are in the map.
- First posted on arXiv 25 Oct 2023.

## Builds on (most cited first, 7 of 7)

- [3D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans](https://mlsanity.com/paper/2002.06289/) (2020)
- [Kaolin: A PyTorch Library for Accelerating 3D Deep Learning Research](https://mlsanity.com/paper/1911.05063/) (2019)
- [Hierarchical Planning for Long-Horizon Manipulation with Geometric and Symbolic Scene Graphs](https://mlsanity.com/paper/2012.07277/) (2020)
- [TASKOGRAPHY: Evaluating robot task planning over large 3D scene graphs](https://mlsanity.com/paper/2207.05006/) (2022)
- [Kineverse: A Symbolic Articulation Model Framework for Model-Agnostic Mobile Manipulation](https://mlsanity.com/paper/2012.05362/) (2020)
- [Understanding URDF: A Survey Based on User Experience](https://mlsanity.com/paper/2302.13442/) (2023)
- [VRKitchen2.0-IndoorKit: A Tutorial for Augmented Indoor Scene Building in Omniverse](https://mlsanity.com/paper/2206.11887/) (2022)

## Built on by (most cited first, 7 of 7)

- [BadRobot: Jailbreaking Embodied LLM Agents in the Physical World](https://mlsanity.com/paper/2407.20242/) (2024)
- [Articulate3D: Holistic Understanding of 3D Scenes as Universal Scene Description](https://mlsanity.com/paper/2412.01398/) (2024)
- [Digital Twin Generation from Visual Data: A Survey](https://mlsanity.com/paper/2504.13159/) (2025)
- [Open, Reproducible and Trustworthy Robot-Based Experiments with Virtual Labs and Digital-Twin-Based Execution Tracing](https://mlsanity.com/paper/2508.11406/) (2025)
- [KnowDemo: Knowledge-Guided Robot Demonstration Generation from Human Videos](https://mlsanity.com/paper/2609.21229/) (2026)
- [Semantic Enrichment of CAD-Based Industrial Environments via Scene Graphs for Simulation and Reasoning](https://mlsanity.com/paper/2601.06415/) (2026)
- [From USD Scenes to Knowledge Graphs: Zero-Shot Ontology Grounding with LLMs](https://mlsanity.com/paper/2606.09134/) (2026)

## Abstract (arXiv)

Robots performing human-scale manipulation tasks require an extensive amount of knowledge about their surroundings in order to perform their actions competently and human-like. In this work, we investigate the use of virtual reality technology as an implementation for robot environment modeling, and present a technique for translating scene graphs into knowledge bases. To this end, we take advantage of the Universal Scene Description (USD) format which is an emerging standard for the authoring, visualization and simulation of complex environments. We investigate the conversion of USD-based environment models into Knowledge Graph (KG) representations that facilitate semantic querying and integration with additional knowledge sources.