mlsanityData to 7 Oct 2026

RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent Geometry and Texture

arXiv 2305.11337 ↗First posted 18 May 2023cs.CVcited by 53

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

The techniques for 3D indoor scene capturing are widely used, but the meshes produced leave much to be desired. In this paper, we propose "RoomDreamer", which leverages powerful natural language to synthesize a new room with a different style. Unlike existing image synthesis methods, our work addresses the challenge of synthesizing both geometry and texture aligned to the input scene structure and prompt simultaneously. The key insight is that a scene should be treated as a whole, taking into account both scene texture and geometry. The proposed framework consists of two significant components: Geometry Guided Diffusion and Mesh Optimization. Geometry Guided Diffusion for 3D Scene guarantees the consistency of the scene style by applying the 2D prior to the entire scene simultaneously. Mesh Optimization improves the geometry and texture jointly and eliminates the artifacts in the scanned scene. To validate the proposed method, real indoor scenes scanned with smartphones are used for extensive experiments, through which the effectiveness of our method is demonstrated.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 40 of its 47 references that are in the map. First posted on arXiv 18 May 2023.

Builds on

40 of 47 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models5 Apr
  2. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  3. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  4. 2022NeRDi: Single-View NeRF Synthesis with Language-Guided Diffusion as General Image Priors6 Dec
  5. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusion2 Dec
  6. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  7. 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Views29 Nov
  8. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  9. 2022RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation17 Nov
  10. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  7. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  8. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  9. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  10. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  11. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 482
  12. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
18 more
  1. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
  2. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 375
  3. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  4. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  5. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  6. 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
  7. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 179
  8. 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
  9. 2021Unconstrained Scene Generation with Locally Conditioned Radiance Fieldscited by 130
  10. 2023Delta Denoising Scorecited by 120
  11. 2018Fast and Flexible Indoor Scene Synthesis via Deep Convolutional Generative Modelscited by 105
  12. 2023DiffCollage: Parallel Generation of Large Content with Diffusion Modelscited by 100
  13. 2019SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scanscited by 77
  14. 2021StyleMesh: Style Transfer for Indoor 3D Scene Reconstructionscited by 58
  15. 2023CC3D: Layout-Conditioned Generation of Compositional 3D Scenescited by 58
  16. 2019Very Long Natural Scenery Image Prediction by Outpaintingcited by 50
  17. 2021RetrievalFuse: Neural 3D Scene Reconstruction with a Databasecited by 28
  18. 2021Plan2Scene: Converting Floorplans to 3D Scenescited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 22; 2025: 19; 2026: 4.

53 papers.

Sort:

By area:

  1. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  2. 2024MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Secondscited by 136
  3. 2024SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Codecited by 125
  4. 2024InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Priorcited by 110
  5. 2023AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homescited by 81
  6. 2024DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splattingcited by 81
  7. 2023PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigationcited by 77
  8. 2023Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraintscited by 59
  9. 2023ControlRoom3D: Room Generation using Semantic Proxy Roomscited by 54
  10. 2023SceneWiz3D: Towards Text-guided 3D Scene Compositioncited by 54
  11. 20244K4DGen: Panoramic 4D Generation at 4K Resolutioncited by 54
  12. 2025CubeDiff: Repurposing Diffusion-Based Image Models for Panorama Generationcited by 35
38 more
  1. 2023DreamSpace: Dreaming Your Room Space with Text-Driven Panoramic Texture Propagationcited by 25
  2. 2024SceneFactor: Factored Latent 3D Diffusion for Controllable 3D Scene Generationcited by 24
  3. 20253D Scene Generation: A Surveycited by 24
  4. 2025DiT360: High-Fidelity Panoramic Image Generation via Hybrid Trainingcited by 24
  5. 2024Pano2Room: Novel View Synthesis from a Single Indoor Panoramacited by 16
  6. 2024Urban Scene Diffusion through Semantic Occupancy Mapcited by 14
  7. 2024RoomTex: Texturing Compositional Indoor Scenes via Iterative Inpaintingcited by 14
  8. 2024SyntheOcc: Synthesize Geometric-Controlled Street View Images through 3D Semantic MPIscited by 14
  9. 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 14
  10. 2024Text-to-3D Shape Generationcited by 13
  11. 2023Recent Advances in Multi-modal 3D Intelligence: A Comprehensive Survey and Evaluationcited by 12
  12. 2024Taming Stable Diffusion for Text to 360deg Panorama Image Generationcited by 12
  13. 2024SceneGenAgent: Precise Industrial Scene Generation with Coding Agentcited by 12
  14. 2024HouseCrafter: Lifting Floorplans to 3D Scenes with 2D Diffusion Modelcited by 9
  15. 2024GenRC: Generative 3D Room Completion from Sparse Image Collectionscited by 9
  16. 2025RoomCraft: Controllable and Complete 3D Indoor Scene Generationcited by 9
  17. 2024Lay-A-Scene: Personalized 3D Object Arrangement Using Text-to-Image Priorscited by 7
  18. 2024InstructLayout: Instruction-Driven 2D and 3D Layout Synthesis with Semantic Graph Priorcited by 6
  19. 2025Towards Geometric and Textural Consistency 3D Scene Generation via Single Image-guided Model Generation and Layout Optimizationcited by 6
  20. 2025From Programs to Poses: Factored Real-World Scene Generation via Learned Program Librariescited by 6
  21. 2025ReSpace: Text-Driven Autoregressive 3D Indoor Scene Synthesis and Editingcited by 5
  22. 2025LatticeWorld: A Multimodal Large Language Model-Empowered Framework for Interactive Complex World Generationcited by 5
  23. 2025MOSAIC: Generating Consistent, Privacy-Preserving Scenes from Multiple Depth Views in Multi-Room Environmentscited by 4
  24. 2025ScanEdit: Hierarchically-Guided Functional 3D Scan Editingcited by 4
  25. 2024SpaceBlender: Creating Context-Rich Collaborative Spaces Through Generative 3D Scene Blendingcited by 3
  26. 2024SM4Depth: Seamless Monocular Metric Depth Estimation across Multiple Cameras and Scenes by One Modelcited by 2
  27. 2024StyleCity: Large-Scale 3D Urban Scenes Stylizationcited by 2
  28. 2024Prim2Room: Layout-Controllable Room Mesh Generation from Primitivescited by 2
  29. 2025Few-step Flow for 3D Generation via Marginal-Data Transport Distillationcited by 2
  30. 2024Style-Consistent 3D Indoor Scene Synthesis with Decoupled Objectscited by 1
  31. 2025Decorum: A Language-Based Approach For Style-Conditioned Synthesis of Indoor 3D Scenescited by 1
  32. 2025SparsyFed: Sparse Adaptive Federated Trainingcited by 1
  33. 2025Programmable-Room: Interactive Textured 3D Room Meshes Generation Empowered by Large Language Modelscited by 1
  34. 2025PrITTI: Primitive-based Generation of Controllable and Editable 3D Semantic Urban Scenescited by 1
  35. 2025MVRoom: Controllable 3D Indoor Scene Generation with Multi-View Diffusion Modelscited by 1
  36. 2026RoamScene3D: Immersive Text-to-3D Scene Generation via Adaptive Object-aware Roamingcited by 1
  37. 2026HOG-Layout: Hierarchical 3D Scene Generation, Optimization and Editing via Vision-Language Modelscited by 1
  38. 2025ESCT3D: Efficient and Selectively Controllable Text-Driven 3D Content Generation with Gaussian Splattingcited by 0

50 most cited of 53 All 53 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis
  2. 2025SemLayoutDiff: Semantic Layout Generation with Diffusion Model for Indoor Scene Synthesis
  3. 2025GeoSceneGraph: Geometric Scene Graph Diffusion Model for Text-guided 3D Indoor Scene Synthesis
  4. 2026Scene Generation at Absolute Scale: Utilizing Semantic and Geometric Guidance From Text for Accurate and Interpretable 3D Indoor Scene Generation
  5. 2024RoomPainter: View-Integrated Diffusion for Consistent Indoor Scene Texturing
  6. 2024SceneCraft: Layout-Guided 3D Scene Generation
  7. 2023SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors
  8. 2025FreeScene: Mixed Graph Diffusion for 3D Scene Synthesis from Free Prompts