# SpatialGrammar: A Domain-Specific Language for LLM-Based 3D Indoor Scene Generation

arXiv: https://arxiv.org/abs/2604.27555 · HTML: https://mlsanity.com/paper/2604.27555/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 30 Apr 2026.

## Builds on (most cited first, 15 of 15)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures](https://mlsanity.com/paper/2211.07600/) (2022)
- [Gibson Env: Real-World Perception for Embodied Agents](https://mlsanity.com/paper/1808.10654/) (2018)
- [BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision](https://mlsanity.com/paper/2211.10439/) (2022)
- [Holodeck: Language Guided Generation of 3D Embodied AI Environments](https://mlsanity.com/paper/2312.09067/) (2023)
- [AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homes](https://mlsanity.com/paper/2312.06644/) (2023)
- [SpatialLM: Training Large Language Models for Structured Indoor Modeling](https://mlsanity.com/paper/2506.07491/) (2025)
- [SceneTeller: Language-to-3D Scene Generation](https://mlsanity.com/paper/2407.20727/) (2024)
- [Global-Local Tree Search in VLMs for 3D Indoor Scene Generation](https://mlsanity.com/paper/2503.18476/) (2025)
- [ArtiScene: Language-Driven Artistic 3D Scene Generation Through Image Intermediary](https://mlsanity.com/paper/2506.00742/) (2025)
- [Direct Numerical Layout Generation for 3D Indoor Scene Synthesis via Spatial Reasoning](https://mlsanity.com/paper/2506.05341/) (2025)
- [FirePlace: Geometric Refinements of LLM Common Sense Reasoning for 3D Object Placement](https://mlsanity.com/paper/2503.04919/) (2025)
- [Procedural Scene Programs for Open-Universe Scene Generation: LLM-Free Error Correction via Program Search](https://mlsanity.com/paper/2510.16147/) (2025)

## Built on by (most cited first, 2 of 2)

- [FuncRoom-Agent: Sequential Feed-Forward 3D Functional Indoor Scene Generation](https://mlsanity.com/paper/2608.29519/) (2026)
- [Learning an Interior Layout Policy in a Domain Specific Language Action Space](https://mlsanity.com/paper/2608.07547/) (2026)

## Abstract (arXiv)

Automatically generating interactive 3D indoor scenes from natural language is crucial for virtual reality, gaming, and embodied AI. However, existing LLM-based approaches often suffer from spatial errors and collisions, in part because common scene representations-raw coordinates or verbose code-are difficult for models to reason about 3D spatial relationships and physical constraints. We propose SpatialGrammar, a domain-specific language that represents gravity-aligned indoor layouts as BEV grid placements with deterministic compilation to valid 3D geometry, enabling verifiable constraint checking. Building on this representation, we develop (1) SG-Agent, a closed-loop system that uses compiler feedback to iteratively refine scenes and enforce collision constraints, and (2) SG-Mini, a 104M-parameter model trained entirely on compiler-validated synthetic data. Across 159 test scenes spanning five scenarios of different complexity, SG-Agent improves spatial fidelity and physical plausibility over prior methods, while SG-Mini performs competitively against larger LLM-based baselines on single-shot generation scenarios.