SceneCode: Executable World Programs for Editable Indoor Scenes with Articulated Objects
Not on the map. Why?
Abstract (arXiv)
Indoor scene synthesis underpins embodied AI, robotic manipulation, and simulation-based policy evaluation, where a useful scene must specify not only what the environment looks like, but also how its objects are structured. Existing pipelines, however, typically represent generated content as static meshes and inherit articulation only from curated asset libraries, which limits object-level controllability and prevents new interactable assets from being produced on demand. We address this gap by formulating physically interactable indoor scene synthesis as programmatic world generation, and present SceneCode, a framework that compiles a natural language prompt into an executable, code-driven indoor world rather than a collection of opaque meshes. A room-level agentic backbone first turns the prompt into a structured house layout and emits per-object AssetRequests through a planner--designer--critic loop. Each request is then routed to one of five code-generation strategies and converted into a synthesized part-wise Blender Python programs that are validated through an execution-guided repair-and-refine loop. The resulting programs are compiled into simulation-ready assets, and exported as SDF for physics simulation. A persistent scene-state registry links object requests, executable programs, rendered geometry, and simulation assets, turning scene assembly into a traceable and locally editable world-building process. We evaluate SceneCode across scene-level synthesis, object-level asset quality, human judgment, and downstream robot interaction. Results show that executable world programs improve prompt-faithful indoor scene generation and produce assets with cleaner mesh structure, and simulator-loadable articulation metadata. Project page: https://scene-code.github.io/.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 32 of its 38 references that are in the map. First posted on arXiv 19 May 2026.
Builds on
32 of 38 references.
References · 32
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2023Visual Instruction Tuningcited by 7,792
- 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
- 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
- 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
- 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 521
- 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
- 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 391
- 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
- 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 337
- 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
20 more
- 2025SAM 3D: 3Dfy Anything in Imagescited by 230
- 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
- 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
- 2023DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesiscited by 159
- 2020SceneFormer: Indoor Scene Generation with Transformerscited by 154
- 2023Infinite Photorealistic Worlds using Procedural Generationcited by 142
- 2024SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Codecited by 125
- 2024InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Priorcited by 110
- 2024URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Imagescited by 106
- 2024LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Modelscited by 94
- 20233D-GPT: Procedural 3D Modeling with Large Language Modelscited by 91
- 2023AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homescited by 81
- 2021AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactionscited by 56
- 2023CAGE: Controllable Articulation GEnerationcited by 50
- 2026SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenescited by 39
- 2024Architect: Generating Vivid and Interactive 3D Scenes with Hierarchical 2D Inpaintingcited by 37
- 2024BlenderLLM: Training Large Language Models for Computer-Aided Design with Self-improvementcited by 26
- 2025SceneEval: Evaluating Semantic Coherence in Text-Conditioned 3D Indoor Scene Synthesiscited by 26
- 2025HSM: Hierarchical Scene Motifs for Multi-Scale Indoor Scene Generationcited by 22
- 2025MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Cloudscited by 15
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memorycited by 7
- 2026WorldClaw: Agentic 3D Open-World Generation at Scalecited by 6
- 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoningcited by 6
- 2026Beyond Placement and Articulation: Usage-Driven Code Scenes for Embodied Interactioncited by 1
- 2026FuncRoom-Agent: Sequential Feed-Forward 3D Functional Indoor Scene Generationcited by 1
- 2026Agentic Visual Generation: From Generative Models to Agentic Controlcited by 0
- 2026Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenescited by 0
- 2026Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes29 Sep 2026
- 2026Agentic Visual Generation: From Generative Models to Agentic Control6 Sep 2026
- 2026Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memory30 Aug 2026
- 2026FuncRoom-Agent: Sequential Feed-Forward 3D Functional Indoor Scene Generation30 Aug 2026
- 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning27 Aug 2026
- 2026Beyond Placement and Articulation: Usage-Driven Code Scenes for Embodied Interaction19 Aug 2026
- 2026WorldClaw: Agentic 3D Open-World Generation at Scale5 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2025SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agent
- 2025Scenethesis: A Language and Vision Agentic Framework for 3D Scene Generation
- 2026Code-as-Room: Generating 3D Rooms from Top-Down View Images via Agentic Code Synthesis
- 2026SceneFoundry: Generating Interactive Infinite 3D Worlds
- 2024Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases
- 2025ReSpace: Text-Driven Autoregressive 3D Indoor Scene Synthesis and Editing
- 2026MANSION: Multi-floor lANguage-to-3D Scene generatIOn for loNg-horizon tasks
- 2026SceneMosaic: Efficient and Diverse Simulation-Ready Scene Generation via Hybrid Agentic Layout Evolution