mlsanityData to 7 Oct 2026

A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspective

arXiv 2405.05173 ↗First posted 8 May 2024cs.CV cs.AI cs.ROcited by 37

Not on the map. Why?

Abstract (arXiv)

3D occupancy perception technology aims to observe and understand dense 3D environments for autonomous vehicles. Owing to its comprehensive perception capability, this technology is emerging as a trend in autonomous driving perception systems, and is attracting significant attention from both industry and academia. Similar to traditional bird's-eye view (BEV) perception, 3D occupancy perception has the nature of multi-source input and the necessity for information fusion. However, the difference is that it captures vertical structures that are ignored by 2D BEV. In this survey, we review the most recent works on 3D occupancy perception, and provide in-depth analyses of methodologies with various input modalities. Specifically, we summarize general network pipelines, highlight information fusion techniques, and discuss effective network training. We evaluate and analyze the occupancy perception performance of the state-of-the-art on the most popular datasets. Furthermore, challenges and future research directions are discussed. We hope this paper will inspire the community and encourage more research work on 3D occupancy perception. A comprehensive list of studies in this survey is publicly available in an active repository that continuously collects the latest work: https://github.com/HuaiyuanXu/3D-Occupancy-Perception.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 126 of its 189 references that are in the map. First posted on arXiv 8 May 2024.

Builds on

126 of 189 references.

References · 126

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2023Segment Anythingcited by 8,158
  9. 2023Visual Instruction Tuningcited by 7,792
  10. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  11. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  12. 2017Squeeze-and-Excitation Networkscited by 4,763
38 more
  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  2. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  4. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  5. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  6. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  7. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  8. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  9. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  10. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  11. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  12. 2022Planning-oriented Autonomous Drivingcited by 1,068
  13. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  14. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  15. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  16. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  17. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  18. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  19. 2021Extract Free Dense Labels from CLIPcited by 580
  20. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  21. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  22. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 510
  23. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
  24. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  25. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  26. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
  27. 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
  28. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  29. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  30. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  31. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 334
  32. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  33. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 310
  34. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 306
  35. 2022Vision Transformer with Deformable Attentioncited by 293
  36. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
  37. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 239
  38. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 230

50 most cited of 126 All 126 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 23; 2026: 9.

37 papers.

Sort:

By area:

  1. 20253D and 4D World Modeling: A Surveycited by 64
  2. 2024GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splattingcited by 50
  3. 2025GaussianFlowOcc: Sparse and Weakly Supervised Occupancy Estimation using Gaussian Splatting and Temporal Flowcited by 24
  4. 2025Doracamom: Joint 3D Detection and Occupancy Prediction with Multi-view 4D Radars and Cameras for Omnidirectional Perceptioncited by 21
  5. 2024Language Driven Occupancy Predictioncited by 17
  6. 2025OccProphet: Pushing Efficiency Frontier of Camera-Only 4D Occupancy Forecasting with Observer-Forecaster-Refiner Frameworkcited by 11
  7. 2024SEED4D: A Synthetic Ego--Exo Dynamic 4D Data Generator, Driving Dataset and Benchmarkcited by 8
  8. 2024LightOcc: Lightweight Spatial Embedding for Efficient Vision-based 3D Occupancy Predictioncited by 6
  9. 2025Efficient Multi-Camera Tokenization with Triplanes for End-to-End Drivingcited by 6
  10. 2025Foundation Models for Autonomous Driving Perception: A Survey Through Core Capabilitiescited by 6
  11. 2025ShelfOcc: Native 3D Supervision beyond LiDAR for Vision-Based Occupancy Estimationcited by 6
  12. 2025OccuFly: A 3D Vision Benchmark for Semantic Scene Completion from the Aerial Perspectivecited by 5
25 more
  1. 2025Inverse++: Vision-Centric 3D Semantic Occupancy Prediction Assisted with 3D Object Detectioncited by 4
  2. 2025From 2D to 3D Cognition: A Brief Survey of General World Modelscited by 4
  3. 2025Progressive Gaussian Transformer with Anisotropy-aware Sampling for Open Vocabulary Occupancy Predictioncited by 4
  4. 2025MetaOcc: Spatio-Temporal Fusion of Surround-View 4D Radar and Camera for 3D Occupancy Prediction with Dual Training Strategiescited by 3
  5. 2025Event-aided Semantic Scene Completioncited by 3
  6. 2025IPFormer: Visual 3D Panoptic Scene Completion with Context-Adaptive Instance Proposalscited by 3
  7. 2025Foundation Models for Trajectory Planning in Autonomous Driving: A Review of Progress and Open Challengescited by 3
  8. 2025MMDrive: Interactive Scene Understanding Beyond Vision with Multi-representational Fusioncited by 3
  9. 2025MinkOcc: Towards real-time label-efficient semantic occupancy predictioncited by 2
  10. 2025MS-Occ: Multi-Stage LiDAR-Camera Fusion for 3D Semantic Occupancy Predictioncited by 2
  11. 2025See through the Dark: Learning Illumination-affined Representations for Nighttime Occupancy Predictioncited by 2
  12. 2025Easy3D-Labels: Supervising Semantic Occupancy Estimation with 3D Pseudo-Labels for Automotive Perceptioncited by 2
  13. 2026Dr.Occ: Depth- and Region-Guided 3D Occupancy from Surround-View Cameras for Autonomous Drivingcited by 2
  14. 2025A Synthetic Benchmark for Collaborative 3D Semantic Occupancy Prediction in V2X-Enabled Autonomous Drivingcited by 1
  15. 2026VGGT-Occ: Geometry-Grounded and Density-Aware Gated Fusion for 3D Occupancy Predictioncited by 1
  16. 2024Neuromorphic spatiotemporal optical flow: Enabling ultrafast visual perception beyond human capabilitiescited by 0
  17. 2025TACOcc:Target-Adaptive Cross-Modal Fusion with Volume Rendering for 3D Semantic Occupancycited by 0
  18. 2025HySafe-AI: Hybrid Safety Architectural Analysis Framework for AI Systems: A Case Studycited by 0
  19. 2026SUG-Occ: Explicit Semantics and Uncertainty Guided Sparse Learning for Efficient 3D Occupancy Predictioncited by 0
  20. 2026GaussianSSC: Triplane-Guided Directional Gaussian Fields for 3D Semantic Completioncited by 0
  21. 2026OccDirector: Language-Guided Behavior and Interaction Generation in 4D Occupancy Spacecited by 0
  22. 2026Hyperbolic Distillation: Geometry-Guided Cross-Modal Transfer for Robust 3D Object Detectioncited by 0
  23. 2026RAF: Reliability-Aware Fusion of Camera, LiDAR, and 4D RADAR for Robust 3D Object Detection in Adverse Weathercited by 0
  24. 2026Group-wise Supervision with Focal-Dice Loss for Long-Tailed Indoor Semantic Occupancy Predictioncited by 0
  25. 2026OccAnyScene: Towards Unified Indoor-Outdoor 3D Occupancy Predictioncited by 0