# Radar Voxel Fusion for 3D Object Detection

arXiv: https://arxiv.org/abs/2106.14087 · HTML: https://mlsanity.com/paper/2106.14087/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 45 references that are in the map.
- First posted on arXiv 26 Jun 2021.

## Builds on (most cited first, 27 of 27)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [PointPainting: Sequential Fusion for 3D Object Detection](https://mlsanity.com/paper/1911.10150/) (2019)
- [Joint 3D Proposal Generation and Object Detection from View Aggregation](https://mlsanity.com/paper/1712.02294/) (2017)
- [Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1706.01307/) (2017)
- [Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/1906.06310/) (2019)
- [CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2011.04841/) (2020)
- [In Defense of Classical Image Processing: Fast Depth Completion on the CPU](https://mlsanity.com/paper/1802.00036/) (2018)
- [A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detection](https://mlsanity.com/paper/2005.07431/) (2020)
- [Monocular Depth Estimation Based On Deep Learning: An Overview](https://mlsanity.com/paper/2003.06620/) (2020)
- [RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects](https://mlsanity.com/paper/2007.14366/) (2020)
- [Distant Vehicle Detection Using Radar and Vision](https://mlsanity.com/paper/1901.10951/) (2019)
- [LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion](https://mlsanity.com/paper/2010.00731/) (2020)
- [Computer Stereo Vision for Autonomous Driving](https://mlsanity.com/paper/2012.03194/) (2020)
- [YOdar: Uncertainty-based Sensor Fusion for Vehicle Detection with Camera and Radar Sensors](https://mlsanity.com/paper/2010.03320/) (2020)
- [Off-the-shelf sensor vs. experimental radar -- How much resolution is necessary in automotive radar classification?](https://mlsanity.com/paper/2006.05485/) (2020)
- [Exploring the Capabilities and Limits of 3D Monocular Object Detection -- A Study on Simulation and Real World Data](https://mlsanity.com/paper/2005.07424/) (2020)

## Built on by (most cited first, 13 of 13)

- [Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2304.10410/) (2023)
- [Improved Orientation Estimation and Detection with Hybrid Object Detection Networks for Automotive Radar](https://mlsanity.com/paper/2205.02111/) (2022)
- [CramNet: Camera-Radar Fusion with Ray-Constrained Cross-Attention for Robust 3D Object Detection](https://mlsanity.com/paper/2210.09267/) (2022)
- [Bi-LRFusion: Bi-Directional LiDAR-Radar Fusion for 3D Dynamic Object Detection](https://mlsanity.com/paper/2306.01438/) (2023)
- [Multi-Sensor Object Anomaly Detection: Unifying Appearance, Geometry, and Internal Properties](https://mlsanity.com/paper/2412.14592/) (2024)
- [Exploring Radar Data Representations in Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2312.04861/) (2023)
- [LiRaFusion: Deep Adaptive LiDAR-Radar Fusion for 3D Object Detection](https://mlsanity.com/paper/2402.11735/) (2024)
- [Milestones in Autonomous Driving and Intelligent Vehicles Part II: Perception and Planning](https://mlsanity.com/paper/2306.01980/) (2023)
- [Self-Supervised Velocity Estimation for Automotive Radar Object Detection Networks](https://mlsanity.com/paper/2207.03146/) (2022)
- [LEROjD: Lidar Extended Radar-Only Object Detection](https://mlsanity.com/paper/2409.05564/) (2024)
- [SoK: Vehicle Orientation Representations for Deep Rotation Estimation](https://mlsanity.com/paper/2112.04421/) (2021)
- [Radar-Lidar Fusion for Object Detection by Designing Effective Convolution Networks](https://mlsanity.com/paper/2310.19405/) (2023)
- [TRACE: Thermal Recognition Attentive-Framework for CO2 Emissions from Livestock](https://mlsanity.com/paper/2604.09648/) (2026)

## Abstract (arXiv)

Automotive traffic scenes are complex due to the variety of possible scenarios, objects, and weather conditions that need to be handled. In contrast to more constrained environments, such as automated underground trains, automotive perception systems cannot be tailored to a narrow field of specific tasks but must handle an ever-changing environment with unforeseen events. As currently no single sensor is able to reliably perceive all relevant activity in the surroundings, sensor data fusion is applied to perceive as much information as possible. Data fusion of different sensors and sensor modalities on a low abstraction level enables the compensation of sensor weaknesses and misdetections among the sensors before the information-rich sensor data are compressed and thereby information is lost after a sensor-individual object detection. This paper develops a low-level sensor fusion network for 3D object detection, which fuses lidar, camera, and radar data. The fusion network is trained and evaluated on the nuScenes data set. On the test set, fusion of radar data increases the resulting AP (Average Precision) detection score by about 5.1% in comparison to the baseline lidar network. The radar sensor fusion proves especially beneficial in inclement conditions such as rain and night scenes. Fusing additional camera data contributes positively only in conjunction with the radar fusion, which shows that interdependencies of the sensors are important for the detection result. Additionally, the paper proposes a novel loss to handle the discontinuity of a simple yaw representation for object detection. Our updated loss increases the detection and orientation estimation performance for all sensor input configurations. The code for this research has been made available on GitHub.