# A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detection

arXiv: https://arxiv.org/abs/2005.07431 · HTML: https://mlsanity.com/paper/2005.07431/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 May 2020.

## Built on by (most cited first, 30 of 70)

- [BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework](https://mlsanity.com/paper/2205.13790/) (2022)
- [FUTR3D: A Unified Sensor Fusion Framework for 3D Detection](https://mlsanity.com/paper/2203.10642/) (2022)
- [Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2304.10410/) (2023)
- [Radar-Camera Pixel Depth Association for Depth Completion](https://mlsanity.com/paper/2106.02778/) (2021)
- [Bridging the View Disparity Between Radar and Camera Features for Multi-modal Fusion 3D Object Detection](https://mlsanity.com/paper/2208.12079/) (2022)
- [R4Dyn: Exploring Radar for Self-Supervised Monocular Depth Estimation of Dynamic Scenes](https://mlsanity.com/paper/2108.04814/) (2021)
- [Depth Estimation from Monocular Images and Sparse radar using Deep Ordinal Regression Network](https://mlsanity.com/paper/2107.07596/) (2021)
- [HRFuser: A Multi-resolution Sensor Fusion Architecture for 2D Object Detection](https://mlsanity.com/paper/2206.15157/) (2022)
- [MVFusion: Multi-View 3D Object Detection with Semantic-aligned Radar and Camera Fusion](https://mlsanity.com/paper/2302.10511/) (2023)
- [Echoes Beyond Points: Unleashing the Power of Raw Radar Data in Multi-modality Fusion](https://mlsanity.com/paper/2307.16532/) (2023)
- [MmWave Radar and Vision Fusion for Object Detection in Autonomous Driving: A Review](https://mlsanity.com/paper/2108.03004/) (2021)
- [Boosting 3D Object Detection by Simulating Multimodality on Point Clouds](https://mlsanity.com/paper/2206.14971/) (2022)
- [Exploring Radar Data Representations in Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2312.04861/) (2023)
- [A Survey and Framework of Cooperative Perception: From Heterogeneous Singleton to Hierarchical Cooperation](https://mlsanity.com/paper/2208.10590/) (2022)
- [DPFT: Dual Perspective Fusion Transformer for Camera-Radar-based Object Detection](https://mlsanity.com/paper/2404.03015/) (2024)
- [Full-Velocity Radar Returns by Radar-Camera Fusion](https://mlsanity.com/paper/2108.10637/) (2021)
- [HydraFusion: Context-Aware Selective Sensor Fusion for Robust and Efficient Autonomous Vehicle Perception](https://mlsanity.com/paper/2201.06644/) (2022)
- [Object Detection in Autonomous Vehicles: Status and Open Challenges](https://mlsanity.com/paper/2201.07706/) (2022)
- [Vision-RADAR fusion for Robotics BEV Detections: A Survey](https://mlsanity.com/paper/2302.06643/) (2023)
- [Radar Voxel Fusion for 3D Object Detection](https://mlsanity.com/paper/2106.14087/) (2021)
- [CaFNet: A Confidence-Driven Framework for Radar Camera Depth Estimation](https://mlsanity.com/paper/2407.00697/) (2024)
- [SAMFusion: Sensor-Adaptive Multimodal Fusion for 3D Object Detection in Adverse Weather](https://mlsanity.com/paper/2508.16408/) (2025)
- [RadSegNet: A Reliable Approach to Radar Camera Fusion](https://mlsanity.com/paper/2208.03849/) (2022)
- [ASY-VRNet: Waterway Panoptic Driving Perception Model based on Asymmetric Fair Fusion of Vision and 4D mmWave Radar](https://mlsanity.com/paper/2308.10287/) (2023)
- [S3: Learnable Sparse Signal Superdensity for Guided Depth Estimation](https://mlsanity.com/paper/2103.02396/) (2021)
- [CR3DT: Camera-RADAR Fusion for 3D Detection and Tracking](https://mlsanity.com/paper/2403.15313/) (2024)
- [Radar Camera Fusion via Representation Learning in Autonomous Driving](https://mlsanity.com/paper/2103.07825/) (2021)
- [Milestones in Autonomous Driving and Intelligent Vehicles Part II: Perception and Planning](https://mlsanity.com/paper/2306.01980/) (2023)
- [Radar and Camera Fusion for Object Detection and Tracking: A Comprehensive Survey](https://mlsanity.com/paper/2410.19872/) (2024)
- [SpaRC: Sparse Radar-Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2411.19860/) (2024)

## Abstract (arXiv)

Object detection in camera images, using deep learning has been proven successfully in recent years. Rising detection rates and computationally efficient network structures are pushing this technique towards application in production vehicles. Nevertheless, the sensor quality of the camera is limited in severe weather conditions and through increased sensor noise in sparsely lit areas and at night. Our approach enhances current 2D object detection networks by fusing camera data and projected sparse radar data in the network layers. The proposed CameraRadarFusionNet (CRF-Net) automatically learns at which level the fusion of the sensor data is most beneficial for the detection result. Additionally, we introduce BlackIn, a training strategy inspired by Dropout, which focuses the learning on a specific sensor type. We show that the fusion network is able to outperform a state-of-the-art image-only network for two different datasets. The code for this research will be made available to the public at: https://github.com/TUMFTM/CameraRadarFusionNet.