mlsanityData to 7 Oct 2026

OccFusion: Multi-Sensor Fusion Framework for 3D Semantic Occupancy Prediction

arXiv 2403.01644 ↗First posted 3 Mar 2024cs.CV cs.ROcited by 32

Not on the map. Why?

Abstract (arXiv)

A comprehensive understanding of 3D scenes is crucial in autonomous vehicles (AVs), and recent models for 3D semantic occupancy prediction have successfully addressed the challenge of describing real-world objects with varied shapes and classes. However, existing methods for 3D occupancy prediction heavily rely on surround-view camera images, making them susceptible to changes in lighting and weather conditions. This paper introduces OccFusion, a novel sensor fusion framework for predicting 3D occupancy. By integrating features from additional sensors, such as lidar and surround view radars, our framework enhances the accuracy and robustness of occupancy prediction, resulting in top-tier performance on the nuScenes benchmark. Furthermore, extensive experiments conducted on the nuScenes and semanticKITTI dataset, including challenging night and rainy scenarios, confirm the superior performance of our sensor fusion strategy across various perception ranges. The code for this framework will be made available at https://github.com/DanielMing123/OccFusion.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 60 of its 73 references that are in the map. First posted on arXiv 3 Mar 2024.

Builds on

60 of 73 references.

References · 60

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  3. 2017Squeeze-and-Excitation Networkscited by 4,763
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  6. 2017Deformable Convolutional Networkscited by 1,481
  7. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  8. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  9. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  10. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  11. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  12. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
38 more
  1. 2020AdaBins: Depth Estimation using Adaptive Binscited by 596
  2. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  3. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  4. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  5. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  6. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 422
  7. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
  8. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  9. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 340
  10. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 334
  11. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  12. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 310
  13. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 306
  14. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  15. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
  16. 2020PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentationcited by 274
  17. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  18. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 222
  19. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 218
  20. 2020Atlas: End-to-End 3D Scene Reconstruction from Posed Imagescited by 206
  21. 2022FUTR3D: A Unified Sensor Fusion Framework for 3D Detectioncited by 194
  22. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  23. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 187
  24. 2021(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Networkcited by 164
  25. 2019MVX-Net: Multimodal VoxelNet for 3D Object Detectioncited by 162
  26. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 162
  27. 2020CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detectioncited by 159
  28. 2022Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?cited by 128
  29. 2020Anisotropic Convolutional Networks for 3D Semantic Scene Completioncited by 126
  30. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 126
  31. 20203D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Priorcited by 124
  32. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 114
  33. 2020Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentationcited by 112
  34. 2023Symphonize 3D Semantic Scene Completion with Contextual Instance Queriescited by 88
  35. 2020AMVNet: Assertion-based Multi-View Fusion Network for LiDAR Semantic Segmentationcited by 87
  36. 2023SparseFusion: Fusing Multi-Modal Sparse Representations for Multi-Sensor 3D Object Detectioncited by 83
  37. 2023Point Cloud Forecasting as a Proxy for 4D Occupancy Forecastingcited by 76
  38. 2022LidarMultiNet: Towards a Unified Multi-Task Network for LiDAR Perceptioncited by 74

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 17; 2026: 10.

32 papers.

Sort:

By area:

  1. 2025Multimodal Fusion and Vision-Language Models: A Survey for Robot Visioncited by 37
  2. 2025Doracamom: Joint 3D Detection and Occupancy Prediction with Multi-view 4D Radars and Cameras for Omnidirectional Perceptioncited by 21
  3. 2024ALOcc: Adaptive Lifting-Based 3D Semantic Occupancy and Cost Volume-Based Flow Predictionscited by 19
  4. 2024LiCROcc: Teach Radar for Accurate Semantic Occupancy Prediction using LiDAR and Cameracited by 18
  5. 2024DAOcc: 3D Object Detection Assisted Multi-Sensor Fusion for 3D Occupancy Predictioncited by 15
  6. 2025GaussianFormer3D: Multi-Modal Gaussian-based Semantic Occupancy Prediction with 3D Deformable Attentioncited by 14
  7. 2024Offboard Occupancy Refinement with Hybrid Propagation for Autonomous Drivingcited by 12
  8. 2025OccCylindrical: Multi-Modal Fusion with Cylindrical Representation for 3D Semantic Occupancy Predictioncited by 5
  9. 2025Humanoid Occupancy: Enabling A Generalized Multimodal Occupancy Perception System on Humanoid Robotscited by 5
  10. 2026GaussianOcc3D: A Gaussian-Based Adaptive Multi-modal 3D Occupancy Predictioncited by 5
  11. 2025Inverse++: Vision-Centric 3D Semantic Occupancy Prediction Assisted with 3D Object Detectioncited by 4
  12. 2025MetaOcc: Spatio-Temporal Fusion of Surround-View 4D Radar and Camera for 3D Occupancy Prediction with Dual Training Strategiescited by 3
20 more
  1. 2025Event-aided Semantic Scene Completioncited by 3
  2. 20254D-ROLLS: 4D Radar Occupancy Learning via LiDAR Supervisioncited by 3
  3. 2025GaussianFusionOcc: A Seamless Sensor Fusion Approach for 3D Occupancy Prediction Using 3D Gaussianscited by 3
  4. 2024A Comprehensive Review of 3D Object Detection in Autonomous Driving: Technological Advances and Future Directionscited by 2
  5. 2025MinkOcc: Towards real-time label-efficient semantic occupancy predictioncited by 2
  6. 2025MS-Occ: Multi-Stage LiDAR-Camera Fusion for 3D Semantic Occupancy Predictioncited by 2
  7. 2026Panoramic Multimodal Semantic Occupancy Prediction for Quadruped Robotscited by 2
  8. 2025PTQAT: A Hybrid Parameter-Efficient Quantization Algorithm for 3D Perception Taskscited by 1
  9. 2025SliceSemOcc: Vertical Slice Based Multimodal 3D Semantic Occupancy Representationcited by 1
  10. 2025REOcc: Camera-Radar Fusion with Radar Feature Enrichment for 3D Occupancy Predictioncited by 1
  11. 2026Humanoid-OmniOcc: Stereo-Based Full-View Occupancy Dataset for Embodied AIcited by 1
  12. 2025OccLE: Label-Efficient 3D Semantic Occupancy Predictioncited by 0
  13. 2025HyGE-Occ: Hybrid View-Transformation with 3D Gaussian and Edge Priors for 3D Panoptic Occupancy Predictioncited by 0
  14. 2026TFusionOcc: T-Primitive Based Object-Centric Multi-Sensor Fusion Framework for 3D Occupancy Predictioncited by 0
  15. 2026M2-Occ: Resilient 3D Semantic Occupancy Prediction for Autonomous Driving with Incomplete Camera Inputscited by 0
  16. 2026Height-Guided Projection Reparameterization for Camera-LiDAR Occupancycited by 0
  17. 2026WeatherOcc3D: VLM-Assisted Adverse Weather Aware 3D Semantic Occupancy Predictioncited by 0
  18. 2026FDR-Occ: Factorized Dense Routing for Full-Spectrum 3D Occupancy Predictioncited by 0
  19. 2026Group-wise Supervision with Focal-Dice Loss for Long-Tailed Indoor Semantic Occupancy Predictioncited by 0
  20. 2026TriO: Tri-Modal Unsupervised Occupancy World Model for Anything Perceptioncited by 0