# DAOcc: 3D Object Detection Assisted Multi-Sensor Fusion for 3D Occupancy Prediction

arXiv: https://arxiv.org/abs/2409.19972 · HTML: https://mlsanity.com/paper/2409.19972/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 62 of its 90 references that are in the map.
- First posted on arXiv 30 Sep 2024.

## Builds on (most cited first, 30 of 62)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models](https://mlsanity.com/paper/2307.05973/) (2023)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection](https://mlsanity.com/paper/2206.10092/) (2022)
- [TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers](https://mlsanity.com/paper/2203.11496/) (2022)
- [Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2302.07817/) (2023)
- [MonoScene: Monocular 3D Semantic Scene Completion](https://mlsanity.com/paper/2112.00726/) (2021)
- [Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection](https://mlsanity.com/paper/1908.09492/) (2019)
- [BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework](https://mlsanity.com/paper/2205.13790/) (2022)
- [Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving](https://mlsanity.com/paper/2304.14365/) (2023)
- [SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving](https://mlsanity.com/paper/2303.09551/) (2023)
- [OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2304.05316/) (2023)
- [DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2203.08195/) (2022)
- [Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion](https://mlsanity.com/paper/2012.03762/) (2020)
- [OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception](https://mlsanity.com/paper/2303.03991/) (2023)
- [Unifying Voxel-based Representation with Transformer for 3D Object Detection](https://mlsanity.com/paper/2206.00630/) (2022)
- [FUTR3D: A Unified Sensor Fusion Framework for 3D Detection](https://mlsanity.com/paper/2203.10642/) (2022)

## Built on by (most cited first, 15 of 15)

- [VoxDet: Rethinking 3D Semantic Occupancy Prediction as Dense Object Detection](https://mlsanity.com/paper/2506.04623/) (2025)
- [Panoramic Multimodal Semantic Occupancy Prediction for Quadruped Robots](https://mlsanity.com/paper/2603.13108/) (2026)
- [Inverse++: Vision-Centric 3D Semantic Occupancy Prediction Assisted with 3D Object Detection](https://mlsanity.com/paper/2504.04732/) (2025)
- [MinkOcc: Towards real-time label-efficient semantic occupancy prediction](https://mlsanity.com/paper/2504.02270/) (2025)
- [GraphGSOcc: Semantic-Geometric Graph Transformer with Dynamic-Static Decoupling for 3D Gaussian Splatting-based Occupancy Prediction](https://mlsanity.com/paper/2506.14825/) (2025)
- [Easy3D-Labels: Supervising Semantic Occupancy Estimation with 3D Pseudo-Labels for Automotive Perception](https://mlsanity.com/paper/2509.26087/) (2025)
- [O3N: Omnidirectional Open-Vocabulary Occupancy Prediction for Embodied Intelligent Robotics](https://mlsanity.com/paper/2603.12144/) (2026)
- [Gau-Occ: Geometry-Completed Gaussians for Multi-Modal 3D Occupancy Prediction](https://mlsanity.com/paper/2603.22852/) (2026)
- [OccLE: Label-Efficient 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2505.20617/) (2025)
- [TFusionOcc: T-Primitive Based Object-Centric Multi-Sensor Fusion Framework for 3D Occupancy Prediction](https://mlsanity.com/paper/2602.06400/) (2026)
- [Height-Guided Projection Reparameterization for Camera-LiDAR Occupancy](https://mlsanity.com/paper/2605.05072/) (2026)
- [Semantic Occupancy Prediction with Dual Range-Voxel Representation](https://mlsanity.com/paper/2606.31688/) (2026)
- [Learning Adaptive Semantic Gaussian Allocation for 3D Occupancy](https://mlsanity.com/paper/2607.21896/) (2026)
- [MATS: A novel multi-modality multi-task learning framework for 3D perception in autonomous driving](https://mlsanity.com/paper/2607.24224/) (2026)
- [OPUS-V2: Bridging the Gap between Sparse Points and Dense Voxels](https://mlsanity.com/paper/2608.29187/) (2026)

## Abstract (arXiv)

Multi-sensor fusion significantly enhances the accuracy and robustness of 3D semantic occupancy prediction, which is crucial for autonomous driving and robotics. However, most existing approaches depend on high-resolution images and complex networks to achieve top performance, hindering their deployment in practical scenarios. Moreover, current multi-sensor fusion approaches mainly focus on improving feature fusion while largely neglecting effective supervision strategies for those features. To address these issues, we propose DAOcc, a novel multi-modal occupancy prediction framework that leverages 3D object detection supervision to assist in achieving superior performance, while using a deployment-friendly image backbone and practical input resolution. In addition, we introduce a BEV View Range Extension strategy to mitigate performance degradation caused by lower image resolution. Extensive experiments demonstrate that DAOcc achieves new state-of-the-art results on both the Occ3D-nuScenes and Occ3D-Waymo benchmarks, and outperforms previous state-of-the-art methods by a significant margin using only a ResNet-50 backbone and 256*704 input resolution. With TensorRT optimization, DAOcc reaches 104.9 FPS while maintaining 54.2 mIoU on an NVIDIA RTX 4090 GPU. Code is available at https://github.com/AlphaPlusTT/DAOcc.