mlsanityData to 7 Oct 2026

Diffusion Probabilistic Models for Scene-Scale 3D Categorical Data

arXiv 2301.00527 ↗First posted 2 Jan 2023cs.CVcited by 33

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

In this paper, we learn a diffusion model to generate 3D data on a scene-scale. Specifically, our model crafts a 3D scene consisting of multiple objects, while recent diffusion research has focused on a single object. To realize our goal, we represent a scene with discrete class labels, i.e., categorical distribution, to assign multiple objects into semantic categories. Thus, we extend discrete diffusion models to learn scene-scale categorical distributions. In addition, we validate that a latent diffusion model can reduce computation costs for training and deploying. To the best of our knowledge, our work is the first to apply discrete and latent diffusion for 3D categorical data on a scene-scale. We further propose to perform semantic scene completion (SSC) by learning a conditional distribution using our diffusion model, where the condition is a partial observation in a sparse point cloud. In experiments, we empirically show that our diffusion models not only generate reasonable scenes, but also perform the scene completion task better than a discriminative model. Our code and models are available at https://github.com/zoomin-lee/scene-scale-diffusion

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 31 of its 32 references that are in the map. First posted on arXiv 2 Jan 2023.

Builds on

31 of 32 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  2. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  3. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Models24 Jan
  4. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 27

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  7. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  8. 2017Neural Discrete Representation Learningcited by 4,650
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  11. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  12. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
15 more
  1. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  2. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  3. 2022GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generationcited by 511
  4. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  5. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 388
  6. 2016OctNet: Learning Deep 3D Representations at High Resolutionscited by 340
  7. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 222
  8. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 187
  9. 2020S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Cloudscited by 102
  10. 2022Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesiscited by 75
  11. 2020Semantic Scene Completion using Local Deep Implicit Functions on LiDAR Datacited by 74
  12. 2021Global Context with Discrete Diffusion in Vector Quantised Modelling for Image Generationcited by 40
  13. 2022MotionSC: Data Set and Network for Real-Time Semantic Mapping in Dynamic Environmentscited by 40
  14. 20193D Semantic Scene Completion from a Single Depth Image using Adversarial Trainingcited by 11
  15. 2021In-N-Out: Towards Good Initialization for Inpainting and Outpaintingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 9; 2025: 12; 2026: 8.

33 papers.

Sort:

By area:

  1. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 183
  2. 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 102
  3. 20253D and 4D World Modeling: A Surveycited by 64
  4. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 59
  5. 2024SemCity: Semantic Scene Generation with Triplane Diffusioncited by 53
  6. 2024LT3SD: Latent Trees for 3D Scene Diffusioncited by 53
  7. 2023Pyramid Diffusion for Fine 3D Large Scene Generationcited by 44
  8. 2024Scaling Diffusion Models to Real-World 3D LiDAR Scene Completioncited by 40
  9. 20253D Scene Generation: A Surveycited by 24
  10. 2025X-Scene: Large-Scale Driving Scene Generation with High Fidelity and Flexible Controllabilitycited by 20
  11. 2025NuiScene: Exploring Efficient Generation of Unbounded Outdoor Scenescited by 19
  12. 2024Distilling Diffusion Models to Efficient 3D LiDAR Scene Completioncited by 10
21 more
  1. 2024OccScene: Semantic Occupancy-based Cross-task Mutual Learning for 3D Scene Generationcited by 10
  2. 2025Controllable 3D Outdoor Scene Generation via Scene Graphscited by 9
  3. 2025Towards Generating Realistic 3D Semantic Training Data for Autonomous Drivingcited by 5
  4. 2025CymbaDiff: Structured Spatial Diffusion for Sketch-based 3D Semantic Urban Scene Generationcited by 3
  5. 2024SSEditor: Controllable Mask-to-Scene Generation with Diffusion Modelcited by 2
  6. 2025Diffusion-Based Generative Models for 3D Occupancy Prediction in Autonomous Drivingcited by 2
  7. 2025Scaffold Diffusion: Sparse Multi-Category Voxel Structure Generation with Discrete Diffusioncited by 2
  8. 2025Scene Coordinate Reconstruction Priorscited by 2
  9. 2026LiFlow: Flow Matching for 3D LiDAR Scene Completioncited by 2
  10. 2024Simultaneous Diffusion Sampling for Conditional LiDAR Generationcited by 1
  11. 2024Map Imagination Like Blind Humans: Group Diffusion Model for Robotic Map Generationcited by 1
  12. 2025DreamTexture: Shape from Virtual Texture with Analysis by Augmentationcited by 1
  13. 20263D-DLP: Self-Supervised 3D Object-Centric Scene Representation Learningcited by 1
  14. 2023Towards Open-World Co-Salient Object Detection with Generative Uncertainty-aware Group Selective Exchange-Maskingcited by 0
  15. 2025PVNet: Point-Voxel Interaction LiDAR Scene Upsampling Via Diffusion Modelscited by 0
  16. 2026EditSSC: Toward Editable Semantic Occupancy Scenes with Unconditional Diffusion Modelscited by 0
  17. 2026Interpretable Human-Label-Free Deep Learning for Real-Bogus Classification with Uncertainty Quantificationcited by 0
  18. 20263D Scene Graph Prediction: Generating Hierarchical Models from Partially Observed Environmentscited by 0
  19. 2026Towards Real-Time and Adaptable LiDAR Scene Completioncited by 0
  20. 2026Generative Semantic Scene Completioncited by 0
  21. 2026SSC-Priors: Exploring Semantic and Visibility Priors to Boost Lidar Semantic Scene Completioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Diffusion Models in 3D Vision: A Survey
  2. 2024Sampling 3D Gaussian Scenes in Seconds with Latent Diffusion Models
  3. 2023Diffusion-SS3D: Diffusion Model for Semi-supervised 3D Object Detection
  4. 2023Diffusion Probabilistic Fields
  5. 2024Diffusion Models Need Visual Priors for Image Generation
  6. 20233D Scene Diffusion Guidance using Scene Graphs
  7. 2026Predicting 3D structure by latent posterior sampling
  8. 2024Joint Generative Modeling of Grounded Scene Graphs and Images via Diffusion Models