# RCM-Fusion: Radar-Camera Multi-Level Fusion for 3D Object Detection

arXiv: https://arxiv.org/abs/2307.10249 · HTML: https://mlsanity.com/paper/2307.10249/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 51 references that are in the map.
- First posted on arXiv 17 Jul 2023.

## Builds on (most cited first, 30 of 42)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries](https://mlsanity.com/paper/2110.06922/) (2021)
- [BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection](https://mlsanity.com/paper/2206.10092/) (2022)
- [PETR: Position Embedding Transformation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2203.05625/) (2022)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection](https://mlsanity.com/paper/2104.10956/) (2021)
- [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection](https://mlsanity.com/paper/1908.09492/) (2019)
- [BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection](https://mlsanity.com/paper/2203.17054/) (2022)
- [BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision](https://mlsanity.com/paper/2211.10439/) (2022)
- [STD: Sparse-to-Dense 3D Object Detector for Point Cloud](https://mlsanity.com/paper/1907.10471/) (2019)
- [Is Pseudo-Lidar needed for Monocular 3D Object detection?](https://mlsanity.com/paper/2108.06417/) (2021)
- [Probabilistic and Geometric Depth: Detecting Objects in Perspective](https://mlsanity.com/paper/2107.14160/) (2021)
- [CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2011.04841/) (2020)
- [M2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representation](https://mlsanity.com/paper/2204.05088/) (2022)
- [PolarFormer: Multi-camera 3D Object Detection with Polar Transformer](https://mlsanity.com/paper/2206.15398/) (2022)

## Built on by (most cited first, 23 of 23)

- [Multimodal Fusion and Vision-Language Models: A Survey for Robot Vision](https://mlsanity.com/paper/2504.02477/) (2025)
- [RadarDistill: Boosting Radar-based Object Detection Performance via Knowledge Distillation from LiDAR Features](https://mlsanity.com/paper/2403.05061/) (2024)
- [CRKD: Enhanced Camera-Radar Object Detection with Cross-modality Knowledge Distillation](https://mlsanity.com/paper/2403.19104/) (2024)
- [LiCROcc: Teach Radar for Accurate Semantic Occupancy Prediction using LiDAR and Camera](https://mlsanity.com/paper/2407.16197/) (2024)
- [Talk2Radar: Bridging Natural Language with 4D mmWave Radar for 3D Referring Expression Comprehension](https://mlsanity.com/paper/2405.12821/) (2024)
- [CRT-Fusion: Camera, Radar, Temporal Fusion Using Motion Information for 3D Object Detection](https://mlsanity.com/paper/2411.03013/) (2024)
- [SpaRC: Sparse Radar-Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2411.19860/) (2024)
- [ClusterFusion: Leveraging Radar Spatial Features for Radar-Camera 3D Object Detection in Autonomous Vehicles](https://mlsanity.com/paper/2309.03734/) (2023)
- [CORENet: Cross-Modal 4D Radar Denoising Network with LiDAR Supervision for Autonomous Driving](https://mlsanity.com/paper/2508.13485/) (2025)
- [A Survey of Deep Learning Based Radar and Vision Fusion for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/2406.00714/) (2024)
- [Integrating Multi-Modal Sensors: A Review of Fusion Techniques for Intelligent Vehicles](https://mlsanity.com/paper/2506.21885/) (2025)
- [RCTDistill: Cross-Modal Knowledge Distillation Framework for Radar-Camera 3D Object Detection with Temporal Fusion](https://mlsanity.com/paper/2509.17712/) (2025)
- [CRAB: Camera-Radar Fusion for Reducing Depth Ambiguity in Backward Projection based View Transformation](https://mlsanity.com/paper/2509.05785/) (2025)
- [Boosting Instance Awareness via Cross-View Correlation with 4D Radar and Camera for 3D Object Detection](https://mlsanity.com/paper/2602.20632/) (2026)
- [FADet: A Multi-sensor 3D Object Detection Network based on Local Featured Attention](https://mlsanity.com/paper/2405.11682/) (2024)
- [SpaRC-AD: A Baseline for Radar-Camera Fusion in End-to-End Autonomous Driving](https://mlsanity.com/paper/2508.10567/) (2025)
- [REOcc: Camera-Radar Fusion with Radar Feature Enrichment for 3D Occupancy Prediction](https://mlsanity.com/paper/2511.06666/) (2025)
- [Control Your Queries: Heterogeneous Query Interaction for Camera-Radar Fusion](https://mlsanity.com/paper/2604.25574/) (2026)
- [REFNet++: Multi-Task Efficient Fusion of Camera and Radar Sensor Data in Bird's-Eye Polar View](https://mlsanity.com/paper/2605.11824/) (2026)
- [Horizon3D: Sparse Radar-Camera Fusion for Long-Range 3D Perception in Autonomous Driving](https://mlsanity.com/paper/2606.31096/) (2026)
- [MATS: A novel multi-modality multi-task learning framework for 3D perception in autonomous driving](https://mlsanity.com/paper/2607.24224/) (2026)
- [RSFusionDet: Underwater RGB-Sonar Multimodal Object Detection](https://mlsanity.com/paper/2608.25367/) (2026)
- [SGDet3D++: Geometry-Grounded Semantics for 4D Radar and Camera 3D Object Detection](https://mlsanity.com/paper/2609.27671/) (2026)

## Abstract (arXiv)

While LiDAR sensors have been successfully applied to 3D object detection, the affordability of radar and camera sensors has led to a growing interest in fusing radars and cameras for 3D object detection. However, previous radar-camera fusion models were unable to fully utilize the potential of radar information. In this paper, we propose Radar-Camera Multi-level fusion (RCM-Fusion), which attempts to fuse both modalities at both feature and instance levels. For feature-level fusion, we propose a Radar Guided BEV Encoder which transforms camera features into precise BEV representations using the guidance of radar Bird's-Eye-View (BEV) features and combines the radar and camera BEV features. For instance-level fusion, we propose a Radar Grid Point Refinement module that reduces localization error by accounting for the characteristics of the radar point clouds. The experiments conducted on the public nuScenes dataset demonstrate that our proposed RCM-Fusion achieves state-of-the-art performances among single frame-based radar-camera fusion methods in the nuScenes 3D object detection benchmark. Code will be made publicly available.