HDSL: A Hierarchical Domain-Specific Language for Structured 3D Indoor Scene Generation and Localized Editing with LLM Agents
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Text-driven indoor scene generation and editing require an intermediate representation that language models can both produce and revise. Existing LLM-based systems often rely on scene graphs or global constraint lists, which are compact but underspecify local geometry and make instruction-based edits difficult to localize. We frame this problem as structured program generation and local program repair, and propose Hierarchical Descriptive Scene Language (HDSL), an XML/CSS-style domain-specific language for structured 3D indoor scenes. HDSL represents rooms, regions, objects, and support surfaces as a tree with local coordinates, making complex scenes easier to plan recursively and easier to retrieve for editing. Our pipeline uses LLM agents to generate HDSL subtrees with bounded verification, grounds non-virtual nodes through multimodal asset retrieval, and applies force-directed layout optimization to repair boundary and collision errors. For editing, Hierarchical Retrieval-Augmented Generation retrieves the relevant subtree, asks the LLM to rewrite only that local context, and merges the result back through a deterministic three-way merge. In our reproduced benchmark, HDSL improves average object coverage, text-scene alignment, and generation time over full text-to-scene baselines while remaining competitive with recent layout-only reproductions on geometry metrics; for editing, HRAG reduces token use by 5.22× and runtime by 6.19×, produces valid DSL for all eight paired edits, and better preserves unrelated scene objects.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 22 of its 32 references that are in the map. First posted on arXiv 8 Jun 2026.
Builds on
22 of 32 references.
References · 22
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
- 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
- 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
- 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 347
- 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 268
- 2023DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesiscited by 166
- 2024PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AIcited by 125
- 2023SatLM: Satisfiability-Aided Language Models Using Declarative Promptingcited by 102
10 more
- 2024I-Design: Personalized LLM Interior Designercited by 87
- 2024LT3SD: Latent Trees for 3D Scene Diffusioncited by 55
- 2025DesignBench: A Comprehensive Benchmark for MLLM-based Front-end Code Generationcited by 45
- 2024The Scene Language: Representing Scenes with Programs, Words, and Embeddingscited by 36
- 2025Direct Numerical Layout Generation for 3D Indoor Scene Synthesis via Spatial Reasoningcited by 28
- 2025HSM: Hierarchical Scene Motifs for Multi-Scale Indoor Scene Generationcited by 24
- 2022Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic Interfacescited by 19
- 2024Arithmetic Reasoning with LLM: Prolog Generation & Permutationcited by 17
- 2025Procedural Scene Programs for Open-Universe Scene Generation: LLM-Free Error Correction via Program Searchcited by 10
- 2025ScanEdit: Hierarchically-Guided Functional 3D Scan Editingcited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025ReSpace: Text-Driven Autoregressive 3D Indoor Scene Synthesis and Editing
- 2026Global Graph-Validated Optimization for VLM-based 3D Indoor Scene Generation
- 2025Decorum: A Language-Based Approach For Style-Conditioned Synthesis of Indoor 3D Scenes
- 2024Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases
- 2026SDesc3D: Towards Layout-Aware 3D Indoor Scene Generation from Short Descriptions
- 2025SceneLCM: End-to-End Layout-Guided Interactive Indoor Scene Generation with Latent Consistency Model
- 2025M3DLayout: A Multi-Source Dataset of 3D Indoor Layouts and Structured Descriptions for 3D Generation
- 2024LLplace: The 3D Indoor Scene Layout Generation and Editing via Large Language Model