mlsanityData to 14 Oct 2026

Pyramid Diffusion for Fine 3D Large Scene Generation

arXiv 2311.12085 ↗First posted 20 Nov 2023cs.CVcited by 44

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Diffusion models have shown remarkable results in generating 2D images and small-scale 3D objects. However, their application to the synthesis of large-scale 3D scenes has been rarely explored. This is mainly due to the inherent complexity and bulky size of 3D scenery data, particularly outdoor scenes, and the limited availability of comprehensive real-world datasets, which makes training a stable scene diffusion model challenging. In this work, we explore how to effectively generate large-scale 3D scenes using the coarse-to-fine paradigm. We introduce a framework, the Pyramid Discrete Diffusion model (PDD), which employs scale-varied diffusion models to progressively generate high-quality outdoor scenes. Experimental results of PDD demonstrate our successful exploration in generating 3D scenes both unconditionally and conditionally. We further showcase the data compatibility of the PDD model, due to its multi-scale architecture: a PDD model trained on one dataset can be easily fine-tuned with another dataset. Code is available at https://github.com/yuhengliu02/pyramid-discrete-diffusion.

Key facts. Cited by 44 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 43 of its 57 references that are in the map. First posted on arXiv 20 Nov 2023.

Builds on

43 of 57 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion14 Nov
  2. 2023Diffusion Probabilistic Models for Scene-Scale 3D Categorical Data2 Jan
  3. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  4. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  5. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 38

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  8. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  10. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  11. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  12. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
26 more
  1. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,476
  2. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  3. 2019A Comprehensive Survey on Transfer Learningcited by 1,160
  4. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  5. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
  6. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 748
  7. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  8. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 248
  9. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  10. 2022PointAvatar: Deformable Point-based Head Avatars from Videoscited by 184
  11. 2023Fast Training of Diffusion Models with Masked Transformerscited by 176
  12. 2020Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Reviewcited by 116
  13. 2022Learning to Generate Realistic LiDAR Point Cloudscited by 93
  14. 2021The Power of Points for Modeling Humans in Clothingcited by 90
  15. 2022InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Imagescited by 82
  16. 2023Generative Diffusion Models on Graphs: Methods and Applicationscited by 81
  17. 2023CityDreamer: Compositional Generative Model of Unbounded 3D Citiescited by 76
  18. 2023SceneDreamer: Unbounded 3D Scene Generation from 2D Image Collectionscited by 72
  19. 2023InfiniCity: Infinite-Scale City Synthesiscited by 68
  20. 2023On the Generalization Properties of Diffusion Modelscited by 68
  21. 2022MotionSC: Data Set and Network for Real-Time Semantic Mapping in Dynamic Environmentscited by 40
  22. 2020Learning 3D Dynamic Scene Representations for Robot Manipulationcited by 39
  23. 2020Campus3D: A Photogrammetry Point Cloud Benchmark for Hierarchical Understanding of Outdoor Scenecited by 27
  24. 2023NPC: Neural Point Characters from Videocited by 26
  25. 2021Learning 3D Dense Correspondence via Canonical Point Autoencodercited by 21
  26. 2019Robust Point Cloud Based Reconstruction of Large-Scale Outdoor Scenescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 21; 2026: 13.

44 papers.

Sort:

By area:

  1. 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 106
  2. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 78
  3. 20253D and 4D World Modeling: A Surveycited by 67
  4. 2024InfiniCube: Unbounded and Controllable Dynamic 3D Driving Scene Generation with World-Guided Video Modelscited by 56
  5. 2024LT3SD: Latent Trees for 3D Scene Diffusioncited by 55
  6. 2024DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenescited by 38
  7. 20253D Scene Generation: A Surveycited by 24
  8. 2024Layout-your-3D: Controllable and Precise 3D Generation with 2D Blueprintcited by 21
  9. 2025X-Scene: Large-Scale Driving Scene Generation with High Fidelity and Flexible Controllabilitycited by 20
  10. 2025NuiScene: Exploring Efficient Generation of Unbounded Outdoor Scenescited by 19
  11. 2025SPATIALGEN: Layout-guided 3D Indoor Scene Generationcited by 14
  12. 2024[MASK] is All You Needcited by 11
32 more
  1. 2025Controllable 3D Outdoor Scene Generation via Scene Graphscited by 9
  2. 2025Conditional Panoramic Image Generation via Masked Autoregressive Modelingcited by 9
  3. 2025Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Methodcited by 8
  4. 2025HoliGS: Holistic Gaussian Splatting for Embodied View Synthesiscited by 7
  5. 2026SceneTransporter: Optimal Transport-Guided Compositional Latent Diffusion for Single-Image Structured 3D Scene Generationcited by 6
  6. 2025Towards Generating Realistic 3D Semantic Training Data for Autonomous Drivingcited by 5
  7. 2025Generating 360deg Video is What You Need For a 3D Scenecited by 5
  8. 2026Extend3D: Town-Scale 3D Generationcited by 5
  9. 2025SyncSDE: A Probabilistic Framework for Diffusion Synchronizationcited by 3
  10. 2025Constructing a 3D Scene from a Single Imagecited by 3
  11. 2025CymbaDiff: Structured Spatial Diffusion for Sketch-based 3D Semantic Urban Scene Generationcited by 3
  12. 2025Self-Evolving 3D Scene Generation from a Single Imagecited by 3
  13. 2024SSEditor: Controllable Mask-to-Scene Generation with Diffusion Modelcited by 2
  14. 2025MSF: Efficient Diffusion Model Via Multi-Scale Latent Factorizecited by 2
  15. 2025Scene Coordinate Reconstruction Priorscited by 2
  16. 2026DVD: Discrete Voxel Diffusion for 3D Generation and Editingcited by 2
  17. 2024Map Imagination Like Blind Humans: Group Diffusion Model for Robotic Map Generationcited by 1
  18. 2025Neural Fidelity Calibration for Informative Sim-to-Real Adaptationcited by 1
  19. 2025PrITTI: Primitive-based Generation of Controllable and Editable 3D Semantic Urban Scenescited by 1
  20. 2026ScenDi: 3D-to-2D Scene Diffusion Cascades for Urban Generationcited by 1
  21. 2026NuiWorld: Exploring a Scalable Framework for End-to-End Controllable World Generationcited by 1
  22. 2026RoamScene3D: Immersive Text-to-3D Scene Generation via Adaptive Object-aware Roamingcited by 1
  23. 2024ArchComplete: Autoregressive 3D Architectural Design Generation with Hierarchical Diffusion-Based Upsamplingcited by 0
  24. 2025WonderVerse: Extendable 3D Scene Generation with Video Generative Modelscited by 0
  25. 2025ASSIST-3D: Adapted Scene Synthesis for Class-Agnostic 3D Instance Segmentationcited by 0
  26. 2026EditSSC: Toward Editable Semantic Occupancy Scenes with Unconditional Diffusion Modelscited by 0
  27. 2026Inclusive Interactive Collisions for Multi-View Consistent Compositional 3D Generationcited by 0
  28. 2026InfiniVerse: Occupancy Guided Unbounded Scene Generation for Autonomous Drivingcited by 0
  29. 2026FPSGen: Flexible Point Cloud Scene Generation with BEV-Supported Transport Flowscited by 0
  30. 2026Generative Semantic Scene Completioncited by 0
  31. 2026GridFlow: Structured Latent Flow for Seamless City-Scale 3D Point Cloud Generationcited by 0
  32. 2026Scaling 3D Generative Priors to Large-Scale Scene Meshes from Multi-View Imagescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation
  2. 2024Denoising Diffusion via Image-Based Rendering
  3. 2024Diffusion Models in 3D Vision: A Survey
  4. 2023HoloDiffusion: Training a 3D Diffusion Model using 2D Images
  5. 2022Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesis
  6. 2025Bolt3D: Generating 3D Scenes in Seconds
  7. 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practices
  8. 2024V3D: Video Diffusion Models are Effective 3D Generators