mlsanityData to 14 Oct 2026

3D Scene Generation: A Survey

arXiv 2505.05474 ↗First posted 8 May 2025cs.CVcited by 24

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

3D scene generation seeks to synthesize spatially structured, semantically meaningful, and photorealistic environments for applications such as immersive media, robotics, autonomous driving, and embodied AI. Early methods based on procedural rules offered scalability but limited diversity. Recent advances in deep generative models (e.g., GANs, diffusion models) and 3D representations (e.g., NeRF, 3D Gaussians) have enabled the learning of real-world scene distributions, improving fidelity, diversity, and view consistency. Recent advances like diffusion models bridge 3D scene synthesis and photorealism by reframing generation as image or video synthesis problems. This survey provides a systematic overview of state-of-the-art approaches, organizing them into four paradigms: procedural generation, neural 3D-based generation, image-based generation, and video-based generation. We analyze their technical foundations, trade-offs, and representative results, and review commonly used datasets, evaluation protocols, and downstream applications. We conclude by discussing key challenges in generation capacity, 3D representation, data and annotations, and evaluation, and outline promising directions including higher fidelity, physics-aware and interactive generation, and unified perception-generation models. This review organizes recent advances in 3D scene generation and highlights promising directions at the intersection of generative AI, 3D vision, and embodied intelligence. To track ongoing developments, we maintain an up-to-date project page: https://github.com/hzxie/Awesome-3D-Scene-Generation.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 308 of its 484 references that are in the map. First posted on arXiv 8 May 2025.

Builds on

308 of 484 references.

Closest ancestors · 47

Same direction, newest first

  1. 2025Scene Splatter: Momentum 3D Scene Generation from Single Image with Video Diffusion Model3 Apr
  2. 2025VideoScene: Distilling Video Diffusion Model to Generate 3D Scenes in One Step2 Apr
  3. 2025Bolt3D: Generating 3D Scenes in Seconds18 Mar
  4. 2025Controllable 3D Outdoor Scene Generation via Scene Graphs10 Mar
  5. 2025CubeDiff: Repurposing Diffusion-Based Image Models for Panorama Generation28 Jan
  6. 2025Layout2Scene: 3D Semantic Layout Guided Scene Generation via Geometry and Appearance Diffusion Priors5 Jan
  7. 2024Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generation30 Dec
  8. 2024RoomPainter: View-Integrated Diffusion for Consistent Indoor Scene Texturing21 Dec
  9. 2024Wonderland: Navigating 3D Scenes from a Single Image16 Dec
  10. 2024DynamicScaler: Seamless and Scalable Video Generation for Panoramic Scenes15 Dec
  11. 2024PaintScene4D: Consistent 4D Scene Generation from Text Prompts5 Dec
  12. 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generation4 Dec
35 more
  1. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models27 Nov
  2. 2024SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting Synthesis25 Nov
  3. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion7 Nov
  4. 2024DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion31 Oct
  5. 2024Diffusion Models in 3D Vision: A Survey7 Oct
  6. 2024DeBaRA: Denoising-Based 3D Room Arrangement Generation26 Sep
  7. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis3 Sep
  8. 2024HoloDreamer: Holistic 3D Panoramic World Generation from Text Descriptions21 Jul
  9. 2024Director3D: Real-world Camera Trajectory and 3D Scene Generation from Text25 Jun
  10. 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Models13 Jun
  11. 20244Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models11 Jun
  12. 2024RoomTex: Texturing Compositional Indoor Scenes via Iterative Inpainting4 Jun
  13. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Models16 May
  14. 2024EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusion2 May
  15. 2024Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpainting30 Apr
  16. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusion10 Apr
  17. 2024Urban Architect: Steerable 3D Urban Scene Generation with Layout Prior10 Apr
  18. 2024Comp4D: LLM-Guided Compositional 4D Scene Generation25 Mar
  19. 2024SemCity: Semantic Scene Generation with Triplane Diffusion12 Mar
  20. 2024360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Model12 Jan
  21. 2023SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors28 Nov
  22. 2023A Unified Approach for Text- and Image-guided 4D Scene Generation28 Nov
  23. 2023Pyramid Diffusion for Fine 3D Large Scene Generation20 Nov
  24. 2023Customizing 360-Degree Panoramas through Text-to-Image Diffusion Models28 Oct
  25. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  26. 2023PanoDiffusion: 360-degree Panorama Outpainting via Diffusion6 Jul
  27. 2023SyncDiffusion: Coherent Montage via Synchronized Joint Diffusions8 Jun
  28. 2023CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusion25 May
  29. 2023Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fields19 May
  30. 2023RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent Geometry and Texture18 May
  31. 2023LDM3D: Latent Diffusion Model for 3D18 May
  32. 2023Consistent View Synthesis with Pose-Guided Diffusion Models30 Mar
  33. 2023DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis24 Mar
  34. 2023CompoNeRF: Text-guided Multi-object Compositional NeRF with Editable 3D Scene Layout24 Mar
  35. 2023Diffusion Probabilistic Models for Scene-Scale 3D Categorical Data2 Jan

Other references · 261

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  8. 2022Generative Adversarial Networkscited by 5,627
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  10. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  11. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
38 more
  1. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  3. 2016Improved Techniques for Training GANscited by 2,970
  4. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  5. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  6. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  7. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  8. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  9. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  10. 2017Improved Training of Wasserstein GANscited by 2,235
  11. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  12. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  13. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  14. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  15. 2018Demystifying MMD GANscited by 1,293
  16. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  17. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,157
  18. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
  19. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  20. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  21. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  22. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
  23. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
  24. 2023Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstructioncited by 759
  25. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
  26. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
  27. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 584
  28. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 575
  29. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 537
  30. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 536
  31. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 523
  32. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  33. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 511
  34. 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 503
  35. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
  36. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 490
  37. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  38. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464

50 most cited of 261 All 261 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 16.

24 papers.

Sort:

By area:

  1. 20253D and 4D World Modeling: A Surveycited by 67
  2. 2026DynamicVLA: A Vision-Language-Action Model for Dynamic Object Manipulationcited by 34
  3. 2025Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Methodcited by 8
  4. 2026WorldClaw: Agentic 3D Open-World Generation at Scalecited by 8
  5. 2025WonderFree: Enhancing Novel View Quality and Cross-View Consistency for 3D Scene Explorationcited by 7
  6. 2025U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequencescited by 7
  7. 2025MajutsuCity: Language-driven Aesthetic-adaptive City Generation with Controllable 3D Assets and Layoutscited by 6
  8. 2026SceneAssistant: A Visual Feedback Agent for Open-Vocabulary 3D Scene Generationcited by 6
  9. 2026One Video, One World: Turning Monocular Video into Physical 4D Scenescited by 6
  10. 2026Imagine a City: CityGenAgent for Procedural 3D City Generationcited by 5
  11. 2026MonoArt: Progressive Structural Reasoning for Monocular Articulated 3D Reconstructioncited by 5
  12. 2026Bridging Semantic and Kinematic Conditions with Diffusion-based Discrete Motion Tokenizercited by 4
12 more
  1. 2025UrbanWorld2.0: A Multimodal Agentic Framework for Reality-Aligned 3D World Generation at City-Scalecited by 3
  2. 2025Complex-Valued Holographic Radiance Fieldscited by 2
  3. 2026SceneOrchestra: Efficient Agentic 3D Scene Synthesis via Full Tool-Call Trajectory Generationcited by 2
  4. 2026GTA: Advancing Image-to-3D World Generation via Geometry Then Appearance Video Diffusioncited by 2
  5. 2026NaLA: A 3D Native LLM Layout Agent for High-quality 3D Scene Generationcited by 2
  6. 2026VibeWorlding: Can Multimodal Agents Construct 3D Open Worlds End-to-End?cited by 2
  7. 2025RecurGS: Interactive Scene Modeling via Discrete-State Recurrent Gaussian Fusioncited by 1
  8. 20263D Generation for Embodied AI and Robotic Simulation: A Surveycited by 1
  9. 2026Embody4D: A Generalist Data Engine for Embodied 4D World Modelingcited by 1
  10. 2026From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Drivingcited by 0
  11. 2026IBRSteG: Learning a Generalizable Steganography Framework for 3D Gaussian Splattingcited by 0
  12. 2026IDEAL-Bench: Indoor Dataset and Evaluation suite for Analyzing 3D Layout reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Recent Advances in 3D Object and Scene Generation: A Survey
  2. 2024Automatic Scene Generation: State-of-the-Art Techniques, Models, Datasets, Challenges, and Future Prospects
  3. 2023NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Models
  4. 2024V3D: Video Diffusion Models are Effective 3D Generators
  5. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models
  6. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior
  7. 2025GEN3D: Generating Domain-Free 3D Scenes from a Single Image
  8. 2023CAD: Photorealistic 3D Generation via Adversarial Distillation