3D Scene Generation: A Survey
On the map: Diffusion models → 3D, point cloud and scene generation
Abstract (arXiv)
3D scene generation seeks to synthesize spatially structured, semantically meaningful, and photorealistic environments for applications such as immersive media, robotics, autonomous driving, and embodied AI. Early methods based on procedural rules offered scalability but limited diversity. Recent advances in deep generative models (e.g., GANs, diffusion models) and 3D representations (e.g., NeRF, 3D Gaussians) have enabled the learning of real-world scene distributions, improving fidelity, diversity, and view consistency. Recent advances like diffusion models bridge 3D scene synthesis and photorealism by reframing generation as image or video synthesis problems. This survey provides a systematic overview of state-of-the-art approaches, organizing them into four paradigms: procedural generation, neural 3D-based generation, image-based generation, and video-based generation. We analyze their technical foundations, trade-offs, and representative results, and review commonly used datasets, evaluation protocols, and downstream applications. We conclude by discussing key challenges in generation capacity, 3D representation, data and annotations, and evaluation, and outline promising directions including higher fidelity, physics-aware and interactive generation, and unified perception-generation models. This review organizes recent advances in 3D scene generation and highlights promising directions at the intersection of generative AI, 3D vision, and embodied intelligence. To track ongoing developments, we maintain an up-to-date project page: https://github.com/hzxie/Awesome-3D-Scene-Generation.
Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 308 of its 484 references that are in the map. First posted on arXiv 8 May 2025.
Builds on
308 of 484 references.
Closest ancestors · 47
Same direction, newest first
- 2025Scene Splatter: Momentum 3D Scene Generation from Single Image with Video Diffusion Model3 Apr
- 2025VideoScene: Distilling Video Diffusion Model to Generate 3D Scenes in One Step2 Apr
- 2025Bolt3D: Generating 3D Scenes in Seconds18 Mar
- 2025Controllable 3D Outdoor Scene Generation via Scene Graphs10 Mar
- 2025CubeDiff: Repurposing Diffusion-Based Image Models for Panorama Generation28 Jan
- 2025Layout2Scene: 3D Semantic Layout Guided Scene Generation via Geometry and Appearance Diffusion Priors5 Jan
- 2024Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generation30 Dec
- 2024RoomPainter: View-Integrated Diffusion for Consistent Indoor Scene Texturing21 Dec
- 2024Wonderland: Navigating 3D Scenes from a Single Image16 Dec
- 2024DynamicScaler: Seamless and Scalable Video Generation for Panoramic Scenes15 Dec
- 2024PaintScene4D: Consistent 4D Scene Generation from Text Prompts5 Dec
- 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generation4 Dec
35 more
- 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models27 Nov
- 2024SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting Synthesis25 Nov
- 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion7 Nov
- 2024DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion31 Oct
- 2024Diffusion Models in 3D Vision: A Survey7 Oct
- 2024DeBaRA: Denoising-Based 3D Room Arrangement Generation26 Sep
- 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis3 Sep
- 2024HoloDreamer: Holistic 3D Panoramic World Generation from Text Descriptions21 Jul
- 2024Director3D: Real-world Camera Trajectory and 3D Scene Generation from Text25 Jun
- 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Models13 Jun
- 20244Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models11 Jun
- 2024RoomTex: Texturing Compositional Indoor Scenes via Iterative Inpainting4 Jun
- 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Models16 May
- 2024EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusion2 May
- 2024Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpainting30 Apr
- 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusion10 Apr
- 2024Urban Architect: Steerable 3D Urban Scene Generation with Layout Prior10 Apr
- 2024Comp4D: LLM-Guided Compositional 4D Scene Generation25 Mar
- 2024SemCity: Semantic Scene Generation with Triplane Diffusion12 Mar
- 2024360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Model12 Jan
- 2023SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors28 Nov
- 2023A Unified Approach for Text- and Image-guided 4D Scene Generation28 Nov
- 2023Pyramid Diffusion for Fine 3D Large Scene Generation20 Nov
- 2023Customizing 360-Degree Panoramas through Text-to-Image Diffusion Models28 Oct
- 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
- 2023PanoDiffusion: 360-degree Panorama Outpainting via Diffusion6 Jul
- 2023SyncDiffusion: Coherent Montage via Synchronized Joint Diffusions8 Jun
- 2023CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusion25 May
- 2023Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fields19 May
- 2023RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent Geometry and Texture18 May
- 2023LDM3D: Latent Diffusion Model for 3D18 May
- 2023Consistent View Synthesis with Pose-Guided Diffusion Models30 Mar
- 2023DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis24 Mar
- 2023CompoNeRF: Text-guided Multi-object Compositional NeRF with Editable 3D Scene Layout24 Mar
- 2023Diffusion Probabilistic Models for Scene-Scale 3D Categorical Data2 Jan
Other references · 261
- 2017Attention Is All You Needcited by 60,361
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2022Generative Adversarial Networkscited by 5,627
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
38 more
- 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
- 2016Improved Techniques for Training GANscited by 2,970
- 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
- 2017Improved Training of Wasserstein GANscited by 2,235
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
- 2018Demystifying MMD GANscited by 1,293
- 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
- 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,157
- 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
- 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
- 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
- 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
- 2023Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstructioncited by 759
- 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
- 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
- 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 584
- 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 575
- 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 537
- 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 536
- 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 523
- 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
- 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 511
- 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 503
- 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
- 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 490
- 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
- 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
50 most cited of 261 All 261 (JSON)
Built on by
Citing papers per year: 2025: 8; 2026: 16.
24 papers.
Sort:
By area:
- 20253D and 4D World Modeling: A Surveycited by 67
- 2026DynamicVLA: A Vision-Language-Action Model for Dynamic Object Manipulationcited by 34
- 2025Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Methodcited by 8
- 2026WorldClaw: Agentic 3D Open-World Generation at Scalecited by 8
- 2025WonderFree: Enhancing Novel View Quality and Cross-View Consistency for 3D Scene Explorationcited by 7
- 2025U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequencescited by 7
- 2025MajutsuCity: Language-driven Aesthetic-adaptive City Generation with Controllable 3D Assets and Layoutscited by 6
- 2026SceneAssistant: A Visual Feedback Agent for Open-Vocabulary 3D Scene Generationcited by 6
- 2026One Video, One World: Turning Monocular Video into Physical 4D Scenescited by 6
- 2026Imagine a City: CityGenAgent for Procedural 3D City Generationcited by 5
- 2026MonoArt: Progressive Structural Reasoning for Monocular Articulated 3D Reconstructioncited by 5
- 2026Bridging Semantic and Kinematic Conditions with Diffusion-based Discrete Motion Tokenizercited by 4
12 more
- 2025UrbanWorld2.0: A Multimodal Agentic Framework for Reality-Aligned 3D World Generation at City-Scalecited by 3
- 2025Complex-Valued Holographic Radiance Fieldscited by 2
- 2026SceneOrchestra: Efficient Agentic 3D Scene Synthesis via Full Tool-Call Trajectory Generationcited by 2
- 2026GTA: Advancing Image-to-3D World Generation via Geometry Then Appearance Video Diffusioncited by 2
- 2026NaLA: A 3D Native LLM Layout Agent for High-quality 3D Scene Generationcited by 2
- 2026VibeWorlding: Can Multimodal Agents Construct 3D Open Worlds End-to-End?cited by 2
- 2025RecurGS: Interactive Scene Modeling via Discrete-State Recurrent Gaussian Fusioncited by 1
- 20263D Generation for Embodied AI and Robotic Simulation: A Surveycited by 1
- 2026Embody4D: A Generalist Data Engine for Embodied 4D World Modelingcited by 1
- 2026From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Drivingcited by 0
- 2026IBRSteG: Learning a Generalizable Steganography Framework for 3D Gaussian Splattingcited by 0
- 2026IDEAL-Bench: Indoor Dataset and Evaluation suite for Analyzing 3D Layout reasoningcited by 0
- 2026VibeWorlding: Can Multimodal Agents Construct 3D Open Worlds End-to-End?15 Aug 2026
- 2026WorldClaw: Agentic 3D Open-World Generation at Scale5 Aug 2026
- 2026IDEAL-Bench: Indoor Dataset and Evaluation suite for Analyzing 3D Layout reasoning3 Jul 2026
- 2026One Video, One World: Turning Monocular Video into Physical 4D Scenes30 Jun 2026
- 2026IBRSteG: Learning a Generalizable Steganography Framework for 3D Gaussian Splatting29 Jun 2026
- 2026NaLA: A 3D Native LLM Layout Agent for High-quality 3D Scene Generation28 Jun 2026
- 2026GTA: Advancing Image-to-3D World Generation via Geometry Then Appearance Video Diffusion13 May 2026
- 2026Embody4D: A Generalist Data Engine for Embodied 4D World Modeling3 May 2026
- 20263D Generation for Embodied AI and Robotic Simulation: A Survey29 Apr 2026
- 2026SceneOrchestra: Efficient Agentic 3D Scene Synthesis via Full Tool-Call Trajectory Generation21 Apr 2026
- 2026MonoArt: Progressive Structural Reasoning for Monocular Articulated 3D Reconstruction19 Mar 2026
- 2026Bridging Semantic and Kinematic Conditions with Diffusion-based Discrete Motion Tokenizer19 Mar 2026
12 more
- 2026From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Driving18 Mar 2026
- 2026SceneAssistant: A Visual Feedback Agent for Open-Vocabulary 3D Scene Generation12 Mar 2026
- 2026Imagine a City: CityGenAgent for Procedural 3D City Generation5 Feb 2026
- 2026DynamicVLA: A Vision-Language-Action Model for Dynamic Object Manipulation29 Jan 2026
- 2025RecurGS: Interactive Scene Modeling via Discrete-State Recurrent Gaussian Fusion20 Dec 2025
- 2025U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequences2 Dec 2025
- 2025MajutsuCity: Language-driven Aesthetic-adaptive City Generation with Controllable 3D Assets and Layouts25 Nov 2025
- 2025UrbanWorld2.0: A Multimodal Agentic Framework for Reality-Aligned 3D World Generation at City-Scale22 Nov 2025
- 2025Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method27 Oct 2025
- 20253D and 4D World Modeling: A Survey4 Sep 2025
- 2025WonderFree: Enhancing Novel View Quality and Cross-View Consistency for 3D Scene Exploration25 Jun 2025
- 2025Complex-Valued Holographic Radiance Fields10 Jun 2025
- 2026WorldClaw: Agentic 3D Open-World Generation at Scalecited by 8
- 2026SceneAssistant: A Visual Feedback Agent for Open-Vocabulary 3D Scene Generationcited by 6
- 2026SceneOrchestra: Efficient Agentic 3D Scene Synthesis via Full Tool-Call Trajectory Generationcited by 2
- 2026NaLA: A 3D Native LLM Layout Agent for High-quality 3D Scene Generationcited by 2
- 2026VibeWorlding: Can Multimodal Agents Construct 3D Open Worlds End-to-End?cited by 2
- 2026VibeWorlding: Can Multimodal Agents Construct 3D Open Worlds End-to-End?15 Aug 2026
- 2026WorldClaw: Agentic 3D Open-World Generation at Scale5 Aug 2026
- 2026NaLA: A 3D Native LLM Layout Agent for High-quality 3D Scene Generation28 Jun 2026
- 2026SceneOrchestra: Efficient Agentic 3D Scene Synthesis via Full Tool-Call Trajectory Generation21 Apr 2026
- 2026SceneAssistant: A Visual Feedback Agent for Open-Vocabulary 3D Scene Generation12 Mar 2026
- 20253D and 4D World Modeling: A Surveycited by 67
- 2025U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequencescited by 7
- 2025MajutsuCity: Language-driven Aesthetic-adaptive City Generation with Controllable 3D Assets and Layoutscited by 6
- 2026Embody4D: A Generalist Data Engine for Embodied 4D World Modelingcited by 1
- 2026Embody4D: A Generalist Data Engine for Embodied 4D World Modeling3 May 2026
- 2025U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequences2 Dec 2025
- 2025MajutsuCity: Language-driven Aesthetic-adaptive City Generation with Controllable 3D Assets and Layouts25 Nov 2025
- 20253D and 4D World Modeling: A Survey4 Sep 2025
- 2025Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Methodcited by 8
- 2025WonderFree: Enhancing Novel View Quality and Cross-View Consistency for 3D Scene Explorationcited by 7
- 2026One Video, One World: Turning Monocular Video into Physical 4D Scenescited by 6
- 2026Imagine a City: CityGenAgent for Procedural 3D City Generationcited by 5
- 2026MonoArt: Progressive Structural Reasoning for Monocular Articulated 3D Reconstructioncited by 5
- 2025UrbanWorld2.0: A Multimodal Agentic Framework for Reality-Aligned 3D World Generation at City-Scalecited by 3
- 2025Complex-Valued Holographic Radiance Fieldscited by 2
- 2025RecurGS: Interactive Scene Modeling via Discrete-State Recurrent Gaussian Fusioncited by 1
- 20263D Generation for Embodied AI and Robotic Simulation: A Surveycited by 1
- 2026IBRSteG: Learning a Generalizable Steganography Framework for 3D Gaussian Splattingcited by 0
- 2026One Video, One World: Turning Monocular Video into Physical 4D Scenes30 Jun 2026
- 2026IBRSteG: Learning a Generalizable Steganography Framework for 3D Gaussian Splatting29 Jun 2026
- 20263D Generation for Embodied AI and Robotic Simulation: A Survey29 Apr 2026
- 2026MonoArt: Progressive Structural Reasoning for Monocular Articulated 3D Reconstruction19 Mar 2026
- 2026Imagine a City: CityGenAgent for Procedural 3D City Generation5 Feb 2026
- 2025RecurGS: Interactive Scene Modeling via Discrete-State Recurrent Gaussian Fusion20 Dec 2025
- 2025UrbanWorld2.0: A Multimodal Agentic Framework for Reality-Aligned 3D World Generation at City-Scale22 Nov 2025
- 2025Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method27 Oct 2025
- 2025WonderFree: Enhancing Novel View Quality and Cross-View Consistency for 3D Scene Exploration25 Jun 2025
- 2025Complex-Valued Holographic Radiance Fields10 Jun 2025
Similar, not cited
Close in wording, no citation either way
- 2025Recent Advances in 3D Object and Scene Generation: A Survey
- 2024Automatic Scene Generation: State-of-the-Art Techniques, Models, Datasets, Challenges, and Future Prospects
- 2023NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Models
- 2024V3D: Video Diffusion Models are Effective 3D Generators
- 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models
- 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior
- 2025GEN3D: Generating Domain-Free 3D Scenes from a Single Image
- 2023CAD: Photorealistic 3D Generation via Adversarial Distillation