mlsanityData to 7 Oct 2026

Scene as Occupancy

arXiv 2306.02851 ↗First posted 5 Jun 2023cs.CV cs.ROcited by 208

Not on the map. Why?

Cited by 60 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Human driver can easily describe the complex traffic scene by visual system. Such an ability of precise perception is essential for driver's planning. To achieve this, a geometry-aware representation that quantizes the physical 3D scene into structured grid map with semantic labels per cell, termed as 3D Occupancy, would be desirable. Compared to the form of bounding box, a key insight behind occupancy is that it could capture the fine-grained details of critical obstacles in the scene, and thereby facilitate subsequent tasks. Prior or concurrent literature mainly concentrate on a single scene completion task, where we might argue that the potential of this occupancy representation might obsess broader impact. In this paper, we propose OccNet, a multi-view vision-centric pipeline with a cascade and temporal voxel decoder to reconstruct 3D occupancy. At the core of OccNet is a general occupancy embedding to represent 3D physical world. Such a descriptor could be applied towards a wide span of driving tasks, including detection, segmentation and planning. To validate the effectiveness of this new representation and our proposed algorithm, we propose OpenOcc, the first dense high-quality 3D occupancy benchmark built on top of nuScenes. Empirical experiments show that there are evident performance gain across multiple tasks, e.g., motion planning could witness a collision rate reduction by 15%-58%, demonstrating the superiority of our method.

Key facts. Cited by 208 papers in the map (data through 7 Oct 2026). Builds on 33 of its 48 references that are in the map. First posted on arXiv 5 Jun 2023.

Builds on

33 of 48 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  5. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  6. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  7. 2022Planning-oriented Autonomous Drivingcited by 1,068
  8. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  9. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  10. 2017Focal Loss for Dense Object Detectioncited by 749
  11. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  12. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
21 more
  1. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  2. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  3. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  4. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
  5. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 388
  6. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  7. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  8. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 340
  9. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 334
  10. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  11. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 306
  12. 2021HDMapNet: An Online HD Map Construction and Evaluation Frameworkcited by 280
  13. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  14. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 218
  15. 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 165
  16. 2022PersFormer: 3D Lane Detection via Perspective Transformer and the OpenLane Benchmarkcited by 140
  17. 2023OccDepth: A Depth-Aware Method for 3D Semantic Scene Completioncited by 85
  18. 2022SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fieldscited by 49
  19. 2022Monocular 3D Object Detection with Depth from Motioncited by 38
  20. 20233D Data Augmentation for Driving Scenes on Cameracited by 12
  21. 2023Sparse Dense Fusion for 3D Object Detectioncited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 25; 2024: 66; 2025: 78; 2026: 39.

208 papers.

Sort:

By area:

  1. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  2. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
  3. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  4. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 230
  5. 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
  6. 2023A Language Agent for Autonomous Drivingcited by 161
  7. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 155
  8. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 146
  9. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  10. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 126
  11. 2025Epona: Autoregressive Diffusion World Model for Autonomous Drivingcited by 115
  12. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 114
38 more
  1. 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 102
  2. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Drivingcited by 97
  3. 2023COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Predictioncited by 94
  4. 2024SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Predictioncited by 94
  5. 2023WoVoGen: World Volume-aware Diffusion for Controllable Multi-camera Driving Scene Generationcited by 93
  6. 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Modelcited by 91
  7. 2023Symphonize 3D Semantic Scene Completion with Contextual Instance Queriescited by 88
  8. 2024GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Predictioncited by 83
  9. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 76
  10. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Predictioncited by 66
  11. 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPTcited by 66
  12. 20253D and 4D World Modeling: A Surveycited by 64
  13. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
  14. 2023Fully Sparse 3D Occupancy Predictioncited by 61
  15. 2025A Survey of World Models for Autonomous Drivingcited by 60
  16. 2023Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applicationscited by 50
  17. 2024GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splattingcited by 50
  18. 2023OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queriescited by 48
  19. 2024Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Drivingcited by 46
  20. 2025Distilling Multi-modal Large Language Models for Autonomous Drivingcited by 46
  21. 2024RenderWorld: World Model with Self-Supervised 3D Labelcited by 44
  22. 2024EmbodiedOcc: Embodied 3D Occupancy Prediction for Vision-based Online Scene Understandingcited by 43
  23. 2025Centaur: Robust End-to-End Autonomous Driving with Test-Time Trainingcited by 41
  24. 2023OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environmentscited by 39
  25. 2024Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Drivingcited by 39
  26. 2024A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspectivecited by 37
  27. 2024ViewFormer: Exploring Spatiotemporal Modeling for Multi-View 3D Occupancy Perception via View-Guided Transformerscited by 36
  28. 2026DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generationcited by 36
  29. 2023BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detectioncited by 35
  30. 2024Unleashing HyDRa: Hybrid Fusion, Depth Consistency and Radar for Unified 3D Perceptioncited by 35
  31. 2023PPAD: Iterative Interactions of Prediction and Planning for End-to-end Autonomous Drivingcited by 33
  32. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
  33. 2024Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehiclescited by 33
  34. 2024DistillNeRF: Perceiving 3D Scenes from Single-Glance Images by Distilling Neural Fields and Foundation Model Featurescited by 33
  35. 2024Producing and Leveraging Online Map Uncertainty in Trajectory Predictioncited by 32
  36. 2024RadarOcc: Robust 3D Occupancy Prediction with 4D Imaging Radarcited by 32
  37. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  38. 2024Vision-based 3D occupancy prediction in autonomous driving: a review and outlookcited by 29

50 most cited of 208 All 208 (JSON)