mlsanityData to 7 Oct 2026

DreamScene: 3D Gaussian-based Text-to-3D Scene Generation via Formation Pattern Sampling

arXiv 2404.03575 ↗First posted 4 Apr 2024cs.CVcited by 72

Not on the map. Why?

Abstract (arXiv)

Text-to-3D scene generation holds immense potential for the gaming, film, and architecture sectors. Despite significant progress, existing methods struggle with maintaining high quality, consistency, and editing flexibility. In this paper, we propose DreamScene, a 3D Gaussian-based novel text-to-3D scene generation framework, to tackle the aforementioned three challenges mainly via two strategies. First, DreamScene employs Formation Pattern Sampling (FPS), a multi-timestep sampling strategy guided by the formation patterns of 3D objects, to form fast, semantically rich, and high-quality representations. FPS uses 3D Gaussian filtering for optimization stability, and leverages reconstruction techniques to generate plausible textures. Second, DreamScene employs a progressive three-stage camera sampling strategy, specifically designed for both indoor and outdoor settings, to effectively ensure object-environment integration and scene-wide 3D consistency. Last, DreamScene enhances scene editing flexibility by integrating objects and environments, enabling targeted adjustments. Extensive experiments validate DreamScene's superiority over current state-of-the-art techniques, heralding its wide-ranging potential for diverse applications. Code and demos will be released at https://dreamscene-project.github.io .

Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). Builds on 41 of its 44 references that are in the map. First posted on arXiv 4 Apr 2024.

Builds on

41 of 44 references.

References · 41

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  7. 2022Classifier-Free Diffusion Guidancecited by 5,717
  8. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  9. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  10. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  11. 2023A Survey of Large Language Modelscited by 2,537
  12. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
29 more
  1. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  2. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  3. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  4. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  5. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  6. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  7. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  8. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  9. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  10. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  11. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  12. 2023LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPScited by 420
  13. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  14. 2023Compact 3D Gaussian Representation for Radiance Fieldcited by 365
  15. 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
  16. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  17. 2024A Survey on 3D Gaussian Splattingcited by 221
  18. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3Dcited by 134
  19. 2023Text-to-3D with Classifier Score Distillationcited by 127
  20. 2023Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fieldscited by 122
  21. 2024GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splattingcited by 104
  22. 2023Compositional 3D Scene Generation using Locally Conditioned Diffusioncited by 101
  23. 2023DreamTime: An Improved Optimization Strategy for Diffusion-Guided 3D Generationcited by 81
  24. 2023Set-the-Scene: Global-Local Training for Generating Controllable NeRF Scenescited by 75
  25. 2023CompoNeRF: Text-guided Multi-object Compositional NeRF with Editable 3D Scene Layoutcited by 57
  26. 2023SceneWiz3D: Towards Text-guided 3D Scene Compositioncited by 54
  27. 2023Text2Immersion: Generative Immersive Scene with 3D Gaussianscited by 43
  28. 2023CG3D: Compositional Generation for Text-to-3D via Gaussian Splattingcited by 42
  29. 2023Text2Scene: Text-driven Indoor Scene Stylization with Part-aware Detailscited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 17; 2025: 43; 2026: 12.

72 papers.

Sort:

By area:

  1. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
  2. 20243D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunitiescited by 77
  3. 2024LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generationcited by 53
  4. 2024InfiniCube: Unbounded and Controllable Dynamic 3D Driving Scene Generation with World-Guided Video Modelscited by 53
  5. 2024Director3D: Real-world Camera Trajectory and 3D Scene Generation from Textcited by 50
  6. 2024SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting Synthesiscited by 34
  7. 2024MVGenMaster: Scaling Multi-View Generation from Any Image via 3D Priors Enhanced Diffusion Modelcited by 31
  8. 2025WonderTurbo: Generating Interactive 3D World in 0.72 Secondscited by 31
  9. 2025WorldExplorer: Towards Generating Fully Navigable 3D Scenescited by 26
  10. 2026VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Controlcited by 26
  11. 2024DynamicScaler: Seamless and Scalable Video Generation for Panoramic Scenescited by 24
  12. 20253D Scene Generation: A Surveycited by 24
38 more
  1. 2024Layout-your-3D: Controllable and Precise 3D Generation with 2D Blueprintcited by 21
  2. 2025WorldCraft: Photo-Realistic 3D World Creation and Customization via LLM Agentscited by 21
  3. 2025A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generationcited by 19
  4. 2025HiScene: Creating Hierarchical 3D Scenes with Isometric View Generationcited by 18
  5. 2024Advancing Extended Reality with 3D Gaussian Splatting: Innovations and Prospectscited by 17
  6. 2025Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagerycited by 16
  7. 2024HDRSplat: Gaussian Splatting for High Dynamic Range 3D Scene Reconstruction from Raw Imagescited by 15
  8. 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 14
  9. 2025Agentic 3D Scene Generation with Spatially Contextualized VLMscited by 14
  10. 2025HOLODECK 2.0: Vision-Language-Guided 3D World Generation with Editingcited by 14
  11. 2025WorldGrow: Generating Infinite 3D Worldcited by 14
  12. 2024Taming Diffusion Prior for Image Super-Resolution with Domain Shift SDEscited by 13
  13. 2025Layout2Scene: 3D Semantic Layout Guided Scene Generation via Geometry and Appearance Diffusion Priorscited by 10
  14. 2024Articulate3D: Holistic Understanding of 3D Scenes as Universal Scene Descriptioncited by 9
  15. 2025Towards Scalable Spatial Intelligence via 2D-to-3D Data Liftingcited by 8
  16. 2025Scaling Sequence-to-Sequence Generative Neural Renderingcited by 8
  17. 2025AnyTask: an Automated Task and Data Generation Framework for Advancing Sim-to-Real Policy Learningcited by 8
  18. 2024FlowDreamer: Exploring High Fidelity Text-to-3D Generation via Rectified Flowcited by 7
  19. 2025Digital Twin Generation from Visual Data: A Surveycited by 7
  20. 2025Towards Geometric and Textural Consistency 3D Scene Generation via Single Image-guided Model Generation and Layout Optimizationcited by 6
  21. 2026SceneTransporter: Optimal Transport-Guided Compositional Latent Diffusion for Single-Image Structured 3D Scene Generationcited by 6
  22. 2025ReSpace: Text-Driven Autoregressive 3D Indoor Scene Synthesis and Editingcited by 5
  23. 2025LSD-3D: Large-Scale 3D Driving Scene Generation with Geometry Groundingcited by 5
  24. 2025LatticeWorld: A Multimodal Large Language Model-Empowered Framework for Interactive Complex World Generationcited by 5
  25. 2026SceneAssistant: A Visual Feedback Agent for Open-Vocabulary 3D Scene Generationcited by 5
  26. 2025ViDAR: Video Diffusion-Aware 4D Reconstruction From Monocular Inputscited by 4
  27. 2025HLG: Comprehensive 3D Room Construction via Hierarchical Layout Generationcited by 4
  28. 2024Text-to-3D Generation by 2D Editingcited by 3
  29. 2025BloomScene: Lightweight Structured 3D Gaussian Splatting for Crossmodal Scene Generationcited by 3
  30. 20253D Software Synthesis Driven by Constraint-Expressive Intermediate Representationcited by 3
  31. 2025GeoWorld: Providing Full-frame Geometry Features to Facilitate 3D Scene Generationcited by 3
  32. 2025ConsDreamer: Advancing Multi-View Consistency for Zero-Shot Text-to-3D Generationcited by 2
  33. 2025DreamAnywhere: Object-Centric Panoramic 3D Scene Generationcited by 2
  34. 2025TabletopGen: Tabletop Scene Generation and Interactive Simulation for Robotic Manipulationcited by 2
  35. 2026WorldMesh: Generating Navigable Multi-Room 3D Scenes via Mesh-Conditioned Image Diffusioncited by 2
  36. 2024MS2Mesh-XR: Multi-modal Sketch-to-Mesh Generation in XR Environmentscited by 1
  37. 2025Hybrid Mesh-Gaussian Representation for Efficient Indoor Scene Reconstructioncited by 1
  38. 2025DreamScene: 3D Gaussian-based End-to-end Text-to-3D Scene Generationcited by 1

50 most cited of 72 All 72 (JSON)