mlsanityData to 13 Oct 2026

CLIP-Layout: Style-Consistent Indoor Scene Synthesis with Semantic Furniture Embedding

arXiv 2303.03565 ↗First posted 7 Mar 2023cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

Indoor scene synthesis involves automatically picking and placing furniture appropriately on a floor plan, so that the scene looks realistic and is functionally plausible. Such scenes can serve as homes for immersive 3D experiences, or be used to train embodied agents. Existing methods for this task rely on labeled categories of furniture, e.g. bed, chair or table, to generate contextually relevant combinations of furniture. Whether heuristic or learned, these methods ignore instance-level visual attributes of objects, and as a result may produce visually less coherent scenes. In this paper, we introduce an auto-regressive scene model which can output instance-level predictions, using general purpose image embedding based on CLIP. This allows us to learn visual correspondences such as matching color and style, and produce more functionally plausible and aesthetically pleasing scenes. Evaluated on the 3D-FRONT dataset, our model achieves SOTA results in scene synthesis and improves auto-completion metrics by over 50%. Moreover, our embedding-based approach enables zero-shot text-guided scene synthesis and editing, which easily generalizes to furniture not seen during training.

Key facts. Cited by 16 papers in the map (data through 13 Oct 2026). Builds on 34 of its 50 references that are in the map. First posted on arXiv 7 Mar 2023.

Builds on

34 of 50 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,106
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  4. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,019
  5. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,994
  6. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,411
  7. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 553
  8. 2021Neural Fields in Visual Computing and Beyondcited by 539
  9. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  10. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 417
  11. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 377
  12. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 371
22 more
  1. 20203D-FUTURE: 3D Furniture shape with TextUREcited by 308
  2. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  3. 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
  4. 2020SceneFormer: Indoor Scene Generation with Transformerscited by 155
  5. 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
  6. 2019Meta-Sim: Learning to Generate Synthetic Datasetscited by 128
  7. 2020Fast Transformers with Clustered Attentioncited by 123
  8. 2018Fast and Flexible Indoor Scene Synthesis via Deep Convolutional Generative Modelscited by 105
  9. 2018Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Datacited by 94
  10. 20223DILG: Irregular Latent Grids for 3D Generative Modelingcited by 92
  11. 2018Human-centric Indoor Scene Synthesis Using Stochastic Grammarcited by 73
  12. 2023SceneDreamer: Unbounded 3D Scene Generation from 2D Image Collectionscited by 72
  13. 2018Deep Generative Modeling for Scene Synthesis via Hybrid Representationscited by 63
  14. 2020Meta-Sim2: Unsupervised Learning of Scene Structure for Synthetic Data Generationcited by 58
  15. 2022MIME: Human-Aware 3D Scene Generationcited by 57
  16. 2019SceneGraphNet: Neural Message Passing for 3D Indoor Scene Augmentationcited by 55
  17. 2020End-to-End Optimization of Scene Layoutcited by 52
  18. 2022LayoutEnhancer: Generating Good Indoor Layouts from Imperfect Datacited by 20
  19. 2020SceneGen: Generative Contextual Scene Augmentation using Scene Graph Priorscited by 11
  20. 2020Fast 3D Indoor Scene Synthesis with Discrete and Exact Layout Pattern Extractioncited by 7
  21. 2022Mutual Scene Synthesis for Mixed Reality Telepresencecited by 3
  22. 2021Contextual Scene Augmentation and Synthesis via GSACNetcited by 2

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 1; 2024: 8; 2025: 5; 2026: 2.

16 papers.

Sort:

By area:

  1. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 339
  2. 2024InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Priorcited by 112
  3. 2024I-Design: Personalized LLM Interior Designercited by 87
  4. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  5. 2024DeBaRA: Denoising-Based 3D Room Arrangement Generationcited by 20
  6. 2024Forest2Seq: Revitalizing Order Prior for Sequential Indoor Scene Synthesiscited by 12
  7. 2024FurniScene: A Large-scale 3D Room Dataset with Intricate Furnishing Scenescited by 10
  8. 2024InstructLayout: Instruction-Driven 2D and 3D Layout Synthesis with Semantic Graph Priorcited by 6
  9. 2024Human-Aware 3D Scene Generation with Spatially-constrained Diffusion Modelscited by 5
  10. 2025ReSpace: Text-Driven Autoregressive 3D Indoor Scene Synthesis and Editingcited by 5
  11. 20253D Software Synthesis Driven by Constraint-Expressive Intermediate Representationcited by 3
  12. 2026SceneNAT: Masked Generative Modeling for Language-Guided Indoor Scene Synthesiscited by 2
4 more
  1. 2025Decorum: A Language-Based Approach For Style-Conditioned Synthesis of Indoor 3D Scenescited by 1
  2. 2025DecoMind: A Generative AI System for Personalized Interior Design Layoutscited by 0
  3. 2025BIM-Native Tokenization for Constraint-Aware Room Layout Synthesiscited by 0
  4. 2026SPHERE: Adaptive VR Indoor Scene Generation via LLM-Enhanced Spatial Preference Learning and Human-in-the-Loop RLcited by 0