# Beyond Placement and Articulation: Usage-Driven Code Scenes for Embodied Interaction

arXiv: https://arxiv.org/abs/2608.18840 · HTML: https://mlsanity.com/paper/2608.18840/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 27 references that are in the map.
- First posted on arXiv 19 Aug 2026.

## Builds on (most cited first, 16 of 16)

- [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [Holodeck: Language Guided Generation of 3D Embodied AI Environments](https://mlsanity.com/paper/2312.09067/) (2023)
- [ATISS: Autoregressive Transformers for Indoor Scene Synthesis](https://mlsanity.com/paper/2110.03675/) (2021)
- [DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis](https://mlsanity.com/paper/2303.14207/) (2023)
- [Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generation](https://mlsanity.com/paper/2406.11824/) (2024)
- [AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homes](https://mlsanity.com/paper/2312.06644/) (2023)
- [SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes](https://mlsanity.com/paper/2602.09153/) (2026)
- [Articraft: An Agentic System for Scalable Articulated 3D Asset Generation](https://mlsanity.com/paper/2605.15187/) (2026)
- [SceneEval: Evaluating Semantic Coherence in Text-Conditioned 3D Indoor Scene Synthesis](https://mlsanity.com/paper/2503.14756/) (2025)
- [HSM: Hierarchical Scene Motifs for Multi-Scale Indoor Scene Generation](https://mlsanity.com/paper/2503.16848/) (2025)
- [MesaTask: Towards Task-Driven Tabletop Scene Generation via 3D Spatial Reasoning](https://mlsanity.com/paper/2509.22281/) (2025)
- [Imaginarium: Vision-guided High-Quality 3D Scene Layout Generation](https://mlsanity.com/paper/2510.15564/) (2025)
- [SceneCode: Executable World Programs for Editable Indoor Scenes with Articulated Objects](https://mlsanity.com/paper/2605.19587/) (2026)
- [Code-as-Room: Generating 3D Rooms from Top-Down View Images via Agentic Code Synthesis](https://mlsanity.com/paper/2605.18451/) (2026)
- [STABLE: Simulation-Ready Tabletop Layout Generation via a Semantics-Physics Dual System](https://mlsanity.com/paper/2605.16137/) (2026)

## Built on by (most cited first, 2 of 2)

- [VPTwin: Real-Sim-Real Video Prediction for Robotic Manipulation Planning](https://mlsanity.com/paper/2609.33104/) (2026)
- [VIGOR: Verifiable Industrial Geometry Optimization and Repair in Scene Programs](https://mlsanity.com/paper/2610.13145/) (2026)

## Abstract (arXiv)

Indoor scene synthesis provides essential environments for embodied AI, robotic manipulation, and simulation-based policy learning. Recent code-based scene generation methods produce editable and extensible environments, yet they remain focused on visual construction and object-level articulation, leaving the functional usage of scenes largely unmodeled. To address this problem, we present RoomWright, an agentic usage-driven framework for generating 3D scenes represented entirely as code for embodied interaction. RoomWright performs usage-driven object reasoning, which treats each anchor as a task centre and admits task-required objects and their affordances. A code agent further enables multi-part interaction by compiling each interaction into a trigger, condition, effect rule that updates structured object states, capturing causal dependencies across objects. Moreover, since manipuland orientation is ambiguous and hard to recover from pixels, RoomWright alleviates this via annotation-informed usage-guided orientation. Extensive experiments demonstrate the effectiveness of our method. The resulting scenes are executable, editable, and simulation-ready, providing interactive environments for embodied AI and policy learning.