SceneNAT: Masked Generative Modeling for Language-Guided Indoor Scene Synthesis
Not on the map. Why?
Abstract (arXiv)
We present SceneNAT, a masked non-autoregressive Transformer for 3D indoor scene synthesis from natural language instructions. It generates complete scenes in a few parallel decoding passes, improving both quality and efficiency over prior methods. SceneNAT is trained via masked modeling over fully discretized representations of both semantic and spatial attributes. By applying a masking strategy at both the attribute level and the instance level, the model can better capture intra-object and inter-object structure. To boost relational reasoning, SceneNAT employs a relational reasoning module (RRM) that captures implicit spatial constraints. By formulating relation modeling as a set prediction task, it extracts structure-aware features to guide the layout generation without explicit sequential parsing. Extensive experiments on 3D-FRONT show that SceneNAT outperforms state-of-the-art autoregressive and diffusion baselines in both semantic compliance and spatial arrangement accuracy while using substantially lower computational cost, enabling high-throughput generation of diverse scenes at scale.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 65 of its 87 references that are in the map. First posted on arXiv 12 Jan 2026.
Builds on
65 of 87 references.
References · 65
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2020End-to-End Object Detection with Transformerscited by 7,414
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2017Neural Discrete Representation Learningcited by 4,712
- 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
- 2022MaskGIT: Masked Generative Image Transformercited by 1,099
- 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
- 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 400
- 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 380
38 more
- 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 380
- 2023An Embodied Generalist Agent in 3D Worldcited by 365
- 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 347
- 20203D-FUTURE: 3D Furniture shape with TextUREcited by 311
- 2016Using the Output Embedding to Improve Language Modelscited by 272
- 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 268
- 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 239
- 2022MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesiscited by 227
- 2022The Role of ImageNet Classes in Fr'echet Inception Distancecited by 213
- 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 200
- 2023OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understandingcited by 184
- 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
- 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 167
- 2023DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesiscited by 166
- 2020SceneFormer: Indoor Scene Generation with Transformerscited by 158
- 2024PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AIcited by 125
- 2024InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Priorcited by 117
- 2018Fast and Flexible Indoor Scene Synthesis via Deep Convolutional Generative Modelscited by 106
- 2020Fully Non-autoregressive Neural Machine Translation: Tricks of the Tradecited by 99
- 2024LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Modelscited by 96
- 2016Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modelingcited by 92
- 2024I-Design: Personalized LLM Interior Designercited by 87
- 2024Classifier-Free Guidance is a Predictor-Correctorcited by 84
- 2023LEGO-Net: Learning Regular Rearrangements of Objects in Roomscited by 76
- 2024Analysis of Classifier-Free Guidance Weight Schedulerscited by 68
- 2024EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusioncited by 67
- 2022Improved Masked Image Generation with Token-Criticcited by 64
- 2024Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesiscited by 56
- 2025SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agentcited by 56
- 2019SceneGraphNet: Neural Message Passing for 3D Indoor Scene Augmentationcited by 55
- 2023CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusioncited by 54
- 2024LLplace: The 3D Indoor Scene Layout Generation and Editing via Large Language Modelcited by 34
- 2024Mixed Diffusion for 3D Indoor Scene Synthesiscited by 32
- 2025Direct Numerical Layout Generation for 3D Indoor Scene Synthesis via Spatial Reasoningcited by 28
- 2025SceneEval: Evaluating Semantic Coherence in Text-Conditioned 3D Indoor Scene Synthesiscited by 26
- 2025HSM: Hierarchical Scene Motifs for Multi-Scale Indoor Scene Generationcited by 24
- 2024Revisiting Non-Autoregressive Transformers for Efficient Image Synthesiscited by 22
- 2022LayoutEnhancer: Generating Good Indoor Layouts from Imperfect Datacited by 20
50 most cited of 65 All 65 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025ReSpace: Text-Driven Autoregressive 3D Indoor Scene Synthesis and Editing
- 2026CasLayout: Cascaded 3D Layout Diffusion for Indoor Scene Synthesis with Implicit Relation Modeling
- 2026SceneCritic: A Symbolic Evaluator for 3D Indoor Scene Synthesis
- 2026SDesc3D: Towards Layout-Aware 3D Indoor Scene Generation from Short Descriptions
- 2025Decorum: A Language-Based Approach For Style-Conditioned Synthesis of Indoor 3D Scenes
- 2026Global Graph-Validated Optimization for VLM-based 3D Indoor Scene Generation
- 2024Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases
- 2026ThinkBLOX: 3D Indoor Scene Generation with Progressive Reasoning