mlsanityData to 14 Oct 2026

OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception

arXiv 2303.03991 ↗First posted 7 Mar 2023cs.CVcited by 221

Not on the map. Why?

Cited by 24 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Semantic occupancy perception is essential for autonomous driving, as automated vehicles require a fine-grained perception of the 3D urban structures. However, existing relevant benchmarks lack diversity in urban scenes, and they only evaluate front-view predictions. Towards a comprehensive benchmarking of surrounding perception algorithms, we propose OpenOccupancy, which is the first surrounding semantic occupancy perception benchmark. In the OpenOccupancy benchmark, we extend the large-scale nuScenes dataset with dense semantic occupancy annotations. Previous annotations rely on LiDAR points superimposition, where some occupancy labels are missed due to sparse LiDAR channels. To mitigate the problem, we introduce the Augmenting And Purifying (AAP) pipeline to ~2x densify the annotations, where ~4000 human hours are involved in the labeling process. Besides, camera-based, LiDAR-based and multi-modal baselines are established for the OpenOccupancy benchmark. Furthermore, considering the complexity of surrounding occupancy perception lies in the computational burden of high-resolution 3D predictions, we propose the Cascade Occupancy Network (CONet) to refine the coarse prediction, which relatively enhances the performance by ~30% than the baseline. We hope the OpenOccupancy benchmark will boost the development of surrounding occupancy perception algorithms.

Key facts. Cited by 221 papers in the map (data through 14 Oct 2026). Builds on 39 of its 54 references that are in the map. First posted on arXiv 7 Mar 2023.

Builds on

39 of 54 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  5. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  6. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  7. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  8. 2015Spatial Transformer Networkscited by 1,429
  9. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  10. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  11. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  12. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
27 more
  1. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 732
  2. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  3. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
  4. 2016Semantic Scene Completion from a Single Depth Imagecited by 472
  5. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
  6. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
  7. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 223
  8. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 188
  9. 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 166
  10. 2020Anisotropic Convolutional Networks for 3D Semantic Scene Completioncited by 127
  11. 20203D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Priorcited by 126
  12. 2020SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instancescited by 123
  13. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradientscited by 122
  14. 2018In Defense of Classical Image Processing: Fast Depth Completion on the CPUcited by 96
  15. 2020Semantic Scene Completion using Local Deep Implicit Functions on LiDAR Datacited by 74
  16. 20213D Semantic Scene Completion: a Surveycited by 74
  17. 2019Efficient Semantic Scene Completion Network with Spatial Group Convolutioncited by 66
  18. 2019RGBD Based Dimensional Decomposition Residual Network for 3D Semantic Scene Completioncited by 55
  19. 2021Semantic Scene Completion via Integrating Instances and Scene in-the-Loopcited by 55
  20. 2018Two Stream 3D Semantic Scene Completioncited by 48
  21. 2019Cascaded Context Pyramid for Full-Resolution 3D Semantic Scene Completioncited by 43
  22. 2020Depth Based Semantic Scene Completion with Position Importance Aware Losscited by 37
  23. 2019ForkNet: Multi-branch Volumetric Semantic Completion from a Single Depth Imagecited by 32
  24. 2020Attention-based Multi-modal Fusion Network for Semantic Scene Completioncited by 28
  25. 2019SynthCity: A large scale synthetic point cloudcited by 26
  26. 2020SCFusion: Real-time Incremental Scene Reconstruction with Semantic Completioncited by 25
  27. 2019EdgeNet: Semantic Scene Completion from a Single RGB-D Imagecited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 29; 2024: 73; 2025: 71; 2026: 48.

221 papers.

Sort:

By area:

  1. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
  2. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 233
  3. 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
  4. 2023Scene as Occupancycited by 209
  5. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 165
  6. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 157
  7. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  8. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 128
  9. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 115
  10. 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 106
  11. 2023COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Predictioncited by 96
  12. 2024SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Predictioncited by 96
38 more
  1. 2023Symphonize 3D Semantic Scene Completion with Contextual Instance Queriescited by 90
  2. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 78
  3. 2023SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Drivingcited by 75
  4. 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Drivingcited by 69
  5. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Predictioncited by 67
  6. 20253D and 4D World Modeling: A Surveycited by 67
  7. 2023Fully Sparse 3D Occupancy Predictioncited by 64
  8. 2025A Survey of World Models for Autonomous Drivingcited by 60
  9. 2024OPUS: Occupancy Prediction Using a Sparse Setcited by 56
  10. 2023PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Predictioncited by 52
  11. 2023Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applicationscited by 52
  12. 2024FastOcc: Accelerating 3D Occupancy Prediction by Fusing the 2D Bird's-Eye View and Perspective Viewcited by 51
  13. 2024Context and Geometry Aware Voxel Transformer for Semantic Scene Completioncited by 50
  14. 2024GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splattingcited by 50
  15. 2023OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queriescited by 48
  16. 2024Co-Occ: Coupling Explicit Feature Fusion with Volume Rendering Regularization for Multi-Modal 3D Semantic Occupancy Predictioncited by 48
  17. 2024EmbodiedOcc: Embodied 3D Occupancy Prediction for Vision-based Online Scene Understandingcited by 45
  18. 2023OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environmentscited by 41
  19. 2024Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Drivingcited by 41
  20. 2023Camera-based 3D Semantic Scene Completion with Sparse Guidance Networkcited by 40
  21. 2025Multimodal Fusion and Vision-Language Models: A Survey for Robot Visioncited by 40
  22. 2024Occupancy as Set of Pointscited by 38
  23. 2024ViewFormer: Exploring Spatiotemporal Modeling for Multi-View 3D Occupancy Perception via View-Guided Transformerscited by 37
  24. 2024A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspectivecited by 37
  25. 2023UniWorld: Autonomous Driving Pre-training via World Modelscited by 33
  26. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
  27. 2024Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehiclescited by 33
  28. 2024RadarOcc: Robust 3D Occupancy Prediction with 4D Imaging Radarcited by 33
  29. 2025OmniNWM: Omniscient Driving Navigation World Modelscited by 33
  30. 2024OccFusion: Multi-Sensor Fusion Framework for 3D Semantic Occupancy Predictioncited by 32
  31. 2024Hierarchical Temporal Context Learning for Camera-based Semantic Scene Completioncited by 32
  32. 2024Monocular Occupancy Prediction for Scalable Indoor Scenescited by 31
  33. 2024GaussianAD: Gaussian-Centric End-to-End Autonomous Drivingcited by 31
  34. 2024OccMamba: Semantic Occupancy Prediction with State Space Modelscited by 30
  35. 2024OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Drivingcited by 29
  36. 2024Vision-based 3D occupancy prediction in autonomous driving: a review and outlookcited by 29
  37. 2025The Role of World Models in Shaping Autonomous Driving: A Comprehensive Surveycited by 29
  38. 2023Detecting As Labeling: Rethinking LiDAR-camera Fusion in 3D Object Detectioncited by 28

50 most cited of 221 All 221 (JSON)