mlsanityData to 14 Oct 2026

Diffusion-Based Generative Models for 3D Occupancy Prediction in Autonomous Driving

arXiv 2505.23115 ↗First posted 29 May 2025cs.CVcited by 2

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Accurately predicting 3D occupancy grids from visual inputs is critical for autonomous driving, but current discriminative methods struggle with noisy data, incomplete observations, and the complex structures inherent in 3D scenes. In this work, we reframe 3D occupancy prediction as a generative modeling task using diffusion models, which learn the underlying data distribution and incorporate 3D scene priors. This approach enhances prediction consistency, noise robustness, and better handles the intricacies of 3D spatial structures. Our extensive experiments show that diffusion-based generative models outperform state-of-the-art discriminative approaches, delivering more realistic and accurate occupancy predictions, especially in occluded or low-visibility regions. Moreover, the improved predictions significantly benefit downstream planning tasks, highlighting the practical advantages of our method for real-world autonomous driving applications.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 44 of its 45 references that are in the map. First posted on arXiv 29 May 2025.

Builds on

44 of 45 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion2 Nov
  2. 2023DrivingDiffusion: Layout-Guided multi-view driving scene video generation with latent diffusion model11 Oct
  3. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Control4 Oct

Other references · 41

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2022Classifier-Free Diffusion Guidancecited by 5,847
  7. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  8. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  9. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  10. 2022Planning-oriented Autonomous Drivingcited by 1,087
  11. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 950
  12. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
29 more
  1. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 575
  2. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  3. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
  4. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 406
  5. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 389
  6. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
  7. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
  8. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 316
  9. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 313
  10. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 293
  11. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 221
  12. 2023Scene as Occupancycited by 209
  13. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 168
  14. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 165
  15. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 128
  16. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 115
  17. 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
  18. 2023BEVControl: Accurately Controlling Street-view Elements with Multi-perspective Consistency via BEV Sketch Layoutcited by 102
  19. 2023FB-BEV: BEV Representation from Forward-Backward View Transformationscited by 98
  20. 2023UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generationcited by 78
  21. 2023PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Predictioncited by 52
  22. 2022SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fieldscited by 49
  23. 2023Diffusion Probabilistic Models for Scene-Scale 3D Categorical Datacited by 33
  24. 2023DiffBEV: Conditional Diffusion Model for Bird's Eye View Perceptioncited by 30
  25. 2024OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Drivingcited by 29
  26. 2023MapPrior: Bird's-Eye View Map Layout Estimation with Generative Modelscited by 21
  27. 2023Denoising Diffusion Semantic Segmentation with Mask Prior Modelingcited by 15
  28. 2024Panacea+: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 5
  29. 2022LOPR: Latent Occupancy PRediction using Generative Modelscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Dr.Occ: Depth- and Region-Guided 3D Occupancy from Surround-View Cameras for Autonomous Drivingcited by 2
  2. 2026ProOOD: Prototype-Guided Out-of-Distribution 3D Occupancy Predictioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Driving
  2. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving
  3. 2024DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Model
  4. 2024Online Diffusion-Based 3D Occupancy Prediction at the Frontier with Probabilistic Map Reconciliation
  5. 2024Drive-1-to-3: Enriching Diffusion Priors for Novel View Synthesis of Real Vehicles
  6. 2025Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving
  7. 2025OccTENS: 3D Occupancy World Model via Temporal Next-Scale Prediction
  8. 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving