mlsanityData to 7 Oct 2026

Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving

arXiv 2304.14365 ↗First posted 27 Apr 2023cs.CVcited by 325

On the map: World models → World models for autonomous driving

Abstract (arXiv)

Robotic perception requires the modeling of both 3D geometry and semantics. Existing methods typically focus on estimating 3D bounding boxes, neglecting finer geometric details and struggling to handle general, out-of-vocabulary objects. 3D occupancy prediction, which estimates the detailed occupancy states and semantics of a scene, is an emerging task to overcome these limitations. To support 3D occupancy prediction, we develop a label generation pipeline that produces dense, visibility-aware labels for any given scene. This pipeline comprises three stages: voxel densification, occlusion reasoning, and image-guided voxel refinement. We establish two benchmarks, derived from the Waymo Open Dataset and the nuScenes Dataset, namely Occ3D-Waymo and Occ3D-nuScenes benchmarks. Furthermore, we provide an extensive analysis of the proposed dataset with various baseline models. Lastly, we propose a new model, dubbed Coarse-to-Fine Occupancy (CTF-Occ) network, which demonstrates superior performance on the Occ3D benchmarks. The code, data, and benchmarks are released at https://tsinghua-mars-lab.github.io/Occ3D/.

Key facts. Cited by 325 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 47 of its 79 references that are in the map. First posted on arXiv 27 Apr 2023.

Builds on

47 of 79 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers31 Mar

Other references · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  5. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  6. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  7. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  8. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  9. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  10. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  11. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  12. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
34 more
  1. 2020Convolutional Occupancy Networkscited by 760
  2. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  3. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
  4. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  5. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
  6. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  7. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  8. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  9. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  10. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 404
  11. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
  12. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  13. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  14. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 310
  15. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 306
  16. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
  17. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  18. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 222
  19. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 218
  20. 2019Disentangling Monocular 3D Object Detectioncited by 215
  21. 2022FUTR3D: A Unified Sensor Fusion Framework for 3D Detectioncited by 194
  22. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 187
  23. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 177
  24. 2020Predicting Semantic Map Representations from Images using Pyramid Occupancy Networkscited by 175
  25. 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 165
  26. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 161
  27. 2020Anisotropic Convolutional Networks for 3D Semantic Scene Completioncited by 126
  28. 2014Learning Rich Features from RGB-D Images for Object Detection and Segmentationcited by 124
  29. 20203D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Priorcited by 124
  30. 2021FCAF3D: Fully Convolutional Anchor-Free 3D Object Detectioncited by 97
  31. 2023OccDepth: A Depth-Aware Method for 3D Semantic Scene Completioncited by 85
  32. 2021Object DGCNN: 3D Object Detection using Dynamic Graphscited by 58
  33. 2019VoteNet: A Deep Learning Label Fusion Method for Multi-Atlas Segmentationcited by 44
  34. 2019SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequencescited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 34; 2024: 98; 2025: 120; 2026: 73.

325 papers.

Sort:

By area:

  1. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 230
  2. 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
  3. 2023EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervisioncited by 216
  4. 2023Scene as Occupancycited by 208
  5. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 162
  6. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 146
  7. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  8. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 126
  9. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 114
  10. 2023FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugincited by 101
  11. 2023COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Predictioncited by 94
  12. 2024SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Predictioncited by 94
38 more
  1. 2023WoVoGen: World Volume-aware Diffusion for Controllable Multi-camera Driving Scene Generationcited by 93
  2. 2023Symphonize 3D Semantic Scene Completion with Contextual Instance Queriescited by 88
  3. 2024GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Predictioncited by 83
  4. 2024OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Drivingcited by 82
  5. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 76
  6. 2023SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Drivingcited by 72
  7. 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Drivingcited by 67
  8. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Predictioncited by 66
  9. 2024A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlookcited by 65
  10. 2025A Comprehensive Survey on World Models for Embodied AIcited by 65
  11. 20253D and 4D World Modeling: A Surveycited by 64
  12. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
  13. 2023Fully Sparse 3D Occupancy Predictioncited by 61
  14. 2025A Survey of World Models for Autonomous Drivingcited by 60
  15. 2024DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Modelcited by 58
  16. 2024OPUS: Occupancy Prediction Using a Sparse Setcited by 56
  17. 2023PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Predictioncited by 51
  18. 2024FastOcc: Accelerating 3D Occupancy Prediction by Fusing the 2D Bird's-Eye View and Perspective Viewcited by 51
  19. 2023Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applicationscited by 50
  20. 2024Context and Geometry Aware Voxel Transformer for Semantic Scene Completioncited by 50
  21. 2024GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splattingcited by 50
  22. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 50
  23. 2023OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queriescited by 48
  24. 2024GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understandingcited by 46
  25. 2024RenderWorld: World Model with Self-Supervised 3D Labelcited by 44
  26. 2024End-to-End Autonomous Driving through V2X Cooperationcited by 42
  27. 2023Camera-based 3D Semantic Scene Completion with Sparse Guidance Networkcited by 40
  28. 2023OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environmentscited by 39
  29. 2024A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspectivecited by 37
  30. 2024Occupancy as Set of Pointscited by 37
  31. 2024DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenescited by 37
  32. 2025Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysiscited by 37
  33. 2024ViewFormer: Exploring Spatiotemporal Modeling for Multi-View 3D Occupancy Perception via View-Guided Transformerscited by 36
  34. 2023BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detectioncited by 35
  35. 2024Unleashing HyDRa: Hybrid Fusion, Depth Consistency and Radar for Unified 3D Perceptioncited by 35
  36. 2025TartanGround: A Large-Scale Dataset for Ground Robot Perception and Navigationcited by 35
  37. 2026SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Drivingcited by 34
  38. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33

50 most cited of 325 All 325 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving
  2. 2024Online Diffusion-Based 3D Occupancy Prediction at the Frontier with Probabilistic Map Reconciliation
  3. 2026FreeOcc: Training-Free Embodied Open-Vocabulary Occupancy Prediction
  4. 2024Fast Occupancy Network
  5. 2026VLMFusionOcc3D: VLM Assisted Multi-Modal 3D Semantic Occupancy Prediction
  6. 2024LOMA: Language-assisted Semantic Occupancy Network via Triplane Mamba
  7. 2026Occ-VLM: Occupancy Grounded Vision Language Model for Indoor Scene Understanding
  8. 2024UnO: Unsupervised Occupancy Fields for Perception and Forecasting