mlsanityData to 14 Oct 2026

OccFusion: Depth Estimation Free Multi-sensor Fusion for 3D Occupancy Prediction

arXiv 2403.05329 ↗First posted 8 Mar 2024cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

3D occupancy prediction based on multi-sensor fusion,crucial for a reliable autonomous driving system, enables fine-grained understanding of 3D scenes. Previous fusion-based 3D occupancy predictions relied on depth estimation for processing 2D image features. However, depth estimation is an ill-posed problem, hindering the accuracy and robustness of these methods. Furthermore, fine-grained occupancy prediction demands extensive computational resources. To address these issues, we propose OccFusion, a depth estimation free multi-modal fusion framework. Additionally, we introduce a generalizable active training method and an active decoder that can be applied to any occupancy prediction model, with the potential to enhance their performance. Experiments conducted on nuScenes-Occupancy and nuScenes-Occ3D demonstrate our framework's superior performance. Detailed ablation studies highlight the effectiveness of each proposed method.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 44 of its 61 references that are in the map. First posted on arXiv 8 Mar 2024.

Builds on

44 of 61 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  5. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  6. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  7. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  8. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  9. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  10. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
  11. 2017Deep Bayesian Active Learning with Image Datacited by 819
  12. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 732
32 more
  1. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  2. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 579
  3. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  4. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 457
  5. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  6. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 432
  7. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
  8. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
  9. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 341
  10. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
  11. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 327
  12. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 316
  13. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 313
  14. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  15. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 293
  16. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  17. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  18. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 223
  19. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 221
  20. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 188
  21. 2021Multimodal Virtual Point 3D Detectioncited by 166
  22. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 165
  23. 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 145
  24. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 128
  25. 2020Anisotropic Convolutional Networks for 3D Semantic Scene Completioncited by 127
  26. 20203D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Priorcited by 126
  27. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 115
  28. 2023OccDepth: A Depth-Aware Method for 3D Semantic Scene Completioncited by 86
  29. 2024POP-3D: Open-Vocabulary 3D Occupancy Prediction from Imagescited by 52
  30. 2023OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queriescited by 48
  31. 2022A Simple Baseline for Multi-Camera 3D Object Detectioncited by 16
  32. 2023UniScene: Multi-Camera Unified Pre-training via 3D Scene Reconstruction for Autonomous Drivingcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 8; 2026: 2.

13 papers.

Sort:

By area:

  1. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 78
  2. 2024DAOcc: 3D Object Detection Assisted Multi-Sensor Fusion for 3D Occupancy Predictioncited by 15
  3. 2025GaussianFormer3D: Multi-Modal Gaussian-based Semantic Occupancy Prediction with 3D Deformable Attentioncited by 14
  4. 2024EFFOcc: Learning Efficient Occupancy Networks from Minimal Labels for Autonomous Drivingcited by 8
  5. 2025Humanoid Occupancy: Enabling A Generalized Multimodal Occupancy Perception System on Humanoid Robotscited by 5
  6. 2025MinkOcc: Towards real-time label-efficient semantic occupancy predictioncited by 2
  7. 2025Panoptic-CUDAL: Rural Australia Point Cloud Dataset in Rainy Conditionscited by 1
  8. 2026Gau-Occ: Geometry-Completed Gaussians for Multi-Modal 3D Occupancy Predictioncited by 1
  9. 2025DGOcc: Depth-aware Global Query-based Network for Monocular 3D Occupancy Predictioncited by 0
  10. 2025Depth as Points: Center Point-based Depth Estimationcited by 0
  11. 2025OccLE: Label-Efficient 3D Semantic Occupancy Predictioncited by 0
  12. 2025OccStress: Stress-Testing the 4D Occupancy Forecasting Chaincited by 0
1 more
  1. 2026ProOOD: Prototype-Guided Out-of-Distribution 3D Occupancy Predictioncited by 0