# Deep Generative Modeling for Scene Synthesis via Hybrid Representations

arXiv: https://arxiv.org/abs/1808.02084 · HTML: https://mlsanity.com/paper/1808.02084/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Aug 2018.

## Built on by (most cited first, 30 of 63)

- [ProcTHOR: Large-Scale Embodied AI Using Procedural Generation](https://mlsanity.com/paper/2206.06994/) (2022)
- [Self-Supervised Pretraining of 3D Features on any Point-Cloud](https://mlsanity.com/paper/2101.02691/) (2021)
- [ATISS: Autoregressive Transformers for Indoor Scene Synthesis](https://mlsanity.com/paper/2110.03675/) (2021)
- [DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis](https://mlsanity.com/paper/2303.14207/) (2023)
- [PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AI](https://mlsanity.com/paper/2404.09465/) (2024)
- [InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Prior](https://mlsanity.com/paper/2402.04717/) (2024)
- [Constrained Graphic Layout Generation via Latent Optimization](https://mlsanity.com/paper/2108.00871/) (2021)
- [AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homes](https://mlsanity.com/paper/2312.06644/) (2023)
- [FoV-NeRF: Foveated Neural Radiance Fields for Virtual Reality](https://mlsanity.com/paper/2103.16365/) (2021)
- [LEGO-Net: Learning Regular Rearrangements of Objects in Rooms](https://mlsanity.com/paper/2301.09629/) (2023)
- [MIME: Human-Aware 3D Scene Generation](https://mlsanity.com/paper/2212.04360/) (2022)
- [RGBD2: Generative Scene Synthesis via Incremental View Inpainting using RGBD Diffusion Models](https://mlsanity.com/paper/2212.05993/) (2022)
- [Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases](https://mlsanity.com/paper/2403.09675/) (2024)
- [ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators](https://mlsanity.com/paper/2108.09432/) (2021)
- [SceneHGN: Hierarchical Graph Networks for 3D Indoor Scene Generation with Fine-Grained Geometry](https://mlsanity.com/paper/2302.10237/) (2023)
- [HPNet: Deep Primitive Segmentation Using Hybrid Representations](https://mlsanity.com/paper/2105.10620/) (2021)
- [Mixed Diffusion for 3D Indoor Scene Synthesis](https://mlsanity.com/paper/2405.21066/) (2024)
- [Scene Synthesis via Uncertainty-Driven Attribute Synchronization](https://mlsanity.com/paper/2108.13499/) (2021)
- [Indoor Scene Generation from a Collection of Semantic-Segmented Depth Images](https://mlsanity.com/paper/2108.09022/) (2021)
- [NAP: Neural 3D Articulation Prior](https://mlsanity.com/paper/2305.16315/) (2023)
- [Learning 3D Scene Priors with 2D Supervision](https://mlsanity.com/paper/2211.14157/) (2022)
- [HSM: Hierarchical Scene Motifs for Multi-Scale Indoor Scene Generation](https://mlsanity.com/paper/2503.16848/) (2025)
- [3D Scene Generation: A Survey](https://mlsanity.com/paper/2505.05474/) (2025)
- [Chat2Layout: Interactive 3D Furniture Layout with a Multimodal LLM](https://mlsanity.com/paper/2407.21333/) (2024)
- [LayoutEnhancer: Generating Good Indoor Layouts from Imperfect Data](https://mlsanity.com/paper/2202.00185/) (2022)
- [DeBaRA: Denoising-Based 3D Room Arrangement Generation](https://mlsanity.com/paper/2409.18336/) (2024)
- [CLIP-Layout: Style-Consistent Indoor Scene Synthesis with Semantic Furniture Embedding](https://mlsanity.com/paper/2303.03565/) (2023)
- [Patch-based 3D Natural Scene Generation from a Single Example](https://mlsanity.com/paper/2304.12670/) (2023)
- [Text2CT: Towards 3D CT Volume Generation from Free-text Descriptions Using Diffusion Model](https://mlsanity.com/paper/2505.04522/) (2025)
- [LUMINOUS: Indoor Scene Generation for Embodied AI Challenges](https://mlsanity.com/paper/2111.05527/) (2021)

## Abstract (arXiv)

We present a deep generative scene modeling technique for indoor environments. Our goal is to train a generative model using a feed-forward neural network that maps a prior distribution (e.g., a normal distribution) to the distribution of primary objects in indoor scenes. We introduce a 3D object arrangement representation that models the locations and orientations of objects, based on their size and shape attributes. Moreover, our scene representation is applicable for 3D objects with different multiplicities (repetition counts), selected from a database. We show a principled way to train this model by combining discriminator losses for both a 3D object arrangement representation and a 2D image-based representation. We demonstrate the effectiveness of our scene representation and the deep learning method on benchmark datasets. We also show the applications of this generative model in scene interpolation and scene completion.