# From USD Scenes to Knowledge Graphs: Zero-Shot Ontology Grounding with LLMs

arXiv: https://arxiv.org/abs/2606.09134 · HTML: https://mlsanity.com/paper/2606.09134/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 12 references that are in the map.
- First posted on arXiv 8 Jun 2026.

## Builds on (most cited first, 8 of 8)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Foundations of the Socio-physical Model of Activities (SOMA) for Autonomous Robotic Agents](https://mlsanity.com/paper/2011.11972/) (2020)
- [Ontologies in Digital Twins: A Systematic Literature Review](https://mlsanity.com/paper/2308.15168/) (2023)
- [Translating Universal Scene Descriptions into Knowledge Graphs for Robotic Environment](https://mlsanity.com/paper/2310.16737/) (2023)
- [Generating Actionable Robot Knowledge Bases by Combining 3D Scene Graphs with Robot Ontologies](https://mlsanity.com/paper/2507.11770/) (2025)
- [Fitting Description Logic Ontologies to ABox and Query Examples](https://mlsanity.com/paper/2508.08007/) (2025)

## Abstract (arXiv)

Constructing knowledge graphs from 3D simulation scenes is essential for robot task reasoning, but the key bottleneck, grounding scene objects to formal ontology classes, still relies on manually curated dictionaries that are brittle and do not generalize across assets. We investigate whether large language models (LLMs) can automate this grounding step for Universal Scene Description (USD) scenes as a zero-shot, training-free alternative. On a kitchen scene (125 objects) with SOMA-HOME Ontology, LLMs achieve 90-96% exact-match accuracy with descriptive names and 49-89% with abbreviated names, substantially outperforming dictionary and embedding baselines. Under fully opaque names, context-augmented prompting recovers up to 48%. Feature ablation reveals that LLMs primarily exploit semantic cues in the scene graph (sibling names and parent paths); anonymizing these cues reduces accuracy to 0-6%, while geometry alone yields only 4-17%.