# CRT-Fusion: Camera, Radar, Temporal Fusion Using Motion Information for 3D Object Detection

arXiv: https://arxiv.org/abs/2411.03013 · HTML: https://mlsanity.com/paper/2411.03013/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 44 references that are in the map.
- First posted on arXiv 5 Nov 2024.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection](https://mlsanity.com/paper/2206.10092/) (2022)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection](https://mlsanity.com/paper/1908.09492/) (2019)
- [BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection](https://mlsanity.com/paper/2203.17054/) (2022)
- [Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection](https://mlsanity.com/paper/2303.11926/) (2023)
- [Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detection](https://mlsanity.com/paper/2210.02443/) (2022)
- [3D-CVF: Generating Joint Camera and LiDAR Features Using Cross-View Spatial Feature Fusion for 3D Object Detection](https://mlsanity.com/paper/2004.12636/) (2020)
- [CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2011.04841/) (2020)
- [SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videos](https://mlsanity.com/paper/2308.09244/) (2023)
- [Offboard 3D Object Detection from Point Cloud Sequences](https://mlsanity.com/paper/2103.05073/) (2021)
- [BEVStereo: Enhancing Depth Estimation in Multi-view 3D Object Detection with Dynamic Temporal Stereo](https://mlsanity.com/paper/2209.10248/) (2022)
- [CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perception](https://mlsanity.com/paper/2304.00670/) (2023)
- [RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detection](https://mlsanity.com/paper/2403.16440/) (2024)
- [3D-MAN: 3D Multi-frame Attention Network for Object Detection](https://mlsanity.com/paper/2103.16054/) (2021)
- [CRAFT: Camera-Radar 3D Object Detection with Spatio-Contextual Fusion Transformer](https://mlsanity.com/paper/2209.06535/) (2022)
- [MPPNet: Multi-Frame Feature Intertwining with Proxy Points for 3D Temporal Object Detection](https://mlsanity.com/paper/2205.05979/) (2022)
- [STS: Surround-view Temporal Stereo for Multi-view 3D Detection](https://mlsanity.com/paper/2208.10145/) (2022)
- [Temporal Enhanced Training of Multi-view 3D Object Detector via Historical Object Prediction](https://mlsanity.com/paper/2304.00967/) (2023)
- [Bridging the View Disparity Between Radar and Camera Features for Multi-modal Fusion 3D Object Detection](https://mlsanity.com/paper/2208.12079/) (2022)
- [An LSTM Approach to Temporal 3D Object Detection in LiDAR Point Clouds](https://mlsanity.com/paper/2007.12392/) (2020)
- [Improved Orientation Estimation and Detection with Hybrid Object Detection Networks for Automotive Radar](https://mlsanity.com/paper/2205.02111/) (2022)
- [SA-BEV: Generating Semantic-Aware Bird's-Eye-View Feature for Multi-view 3D Object Detection](https://mlsanity.com/paper/2307.11477/) (2023)

## Built on by (most cited first, 18 of 18)

- [Doracamom: Joint 3D Detection and Occupancy Prediction with Multi-view 4D Radars and Cameras for Omnidirectional Perception](https://mlsanity.com/paper/2501.15394/) (2025)
- [OmniVLA: Physically-Grounded Multimodal VLA with Unified Multi-Sensor Perception for Robotic Manipulation](https://mlsanity.com/paper/2511.01210/) (2025)
- [A Survey of Multi-sensor Fusion Perception for Embodied AI: Background, Methods, Challenges and Prospects](https://mlsanity.com/paper/2506.19769/) (2025)
- [RCTDistill: Cross-Modal Knowledge Distillation Framework for Radar-Camera 3D Object Detection with Temporal Fusion](https://mlsanity.com/paper/2509.17712/) (2025)
- [RQR3D: Reparametrizing the regression targets for BEV-based 3D object detection](https://mlsanity.com/paper/2505.17732/) (2025)
- [RISE: Single Static Radar-based Indoor Scene Understanding](https://mlsanity.com/paper/2511.14019/) (2025)
- [MuseVLA: An Adaptive Multimodal Sensing Vision-Language-Action Model for Robotic Manipulation](https://mlsanity.com/paper/2606.17598/) (2026)
- [SpaRC-AD: A Baseline for Radar-Camera Fusion in End-to-End Autonomous Driving](https://mlsanity.com/paper/2508.10567/) (2025)
- [REOcc: Camera-Radar Fusion with Radar Feature Enrichment for 3D Occupancy Prediction](https://mlsanity.com/paper/2511.06666/) (2025)
- [Revisiting Radar Camera Alignment by Contrastive Learning for 3D Object Detection](https://mlsanity.com/paper/2504.16368/) (2025)
- [IMKD: Intensity-Aware Multi-Level Knowledge Distillation for Camera-Radar Fusion](https://mlsanity.com/paper/2512.15581/) (2025)
- [AdaRadar: Rate Adaptive Spectral Compression for Radar-based Perception](https://mlsanity.com/paper/2603.17979/) (2026)
- [Radar-Camera BEV Multi-Task Learning with Cross-Task Attention Bridge for Joint 3D Detection and Segmentation](https://mlsanity.com/paper/2604.12918/) (2026)
- [Control Your Queries: Heterogeneous Query Interaction for Camera-Radar Fusion](https://mlsanity.com/paper/2604.25574/) (2026)
- [Horizon3D: Sparse Radar-Camera Fusion for Long-Range 3D Perception in Autonomous Driving](https://mlsanity.com/paper/2606.31096/) (2026)
- [CRISP: A Spatiotemporal Camera-Radar Backbone for Driving via Forecasting-Based World-Model Pretraining](https://mlsanity.com/paper/2607.04541/) (2026)
- [RLG-TPV: Radar- and LiDAR-Guided Tri-Perspective View Fusion for Camera-Radar 3D Object Detection](https://mlsanity.com/paper/2608.29194/) (2026)
- [If It Moves, Radar Knows: A Physics-Aware Radar Transformer for Class-Agnostic Moving-Object Detection](https://mlsanity.com/paper/2609.02289/) (2026)

## Abstract (arXiv)

Accurate and robust 3D object detection is a critical component in autonomous vehicles and robotics. While recent radar-camera fusion methods have made significant progress by fusing information in the bird's-eye view (BEV) representation, they often struggle to effectively capture the motion of dynamic objects, leading to limited performance in real-world scenarios. In this paper, we introduce CRT-Fusion, a novel framework that integrates temporal information into radar-camera fusion to address this challenge. Our approach comprises three key modules: Multi-View Fusion (MVF), Motion Feature Estimator (MFE), and Motion Guided Temporal Fusion (MGTF). The MVF module fuses radar and image features within both the camera view and bird's-eye view, thereby generating a more precise unified BEV representation. The MFE module conducts two simultaneous tasks: estimation of pixel-wise velocity information and BEV segmentation. Based on the velocity and the occupancy score map obtained from the MFE module, the MGTF module aligns and fuses feature maps across multiple timestamps in a recurrent manner. By considering the motion of dynamic objects, CRT-Fusion can produce robust BEV feature maps, thereby improving detection accuracy and robustness. Extensive evaluations on the challenging nuScenes dataset demonstrate that CRT-Fusion achieves state-of-the-art performance for radar-camera-based 3D object detection. Our approach outperforms the previous best method in terms of NDS by +1.7%, while also surpassing the leading approach in mAP by +1.4%. These significant improvements in both metrics showcase the effectiveness of our proposed fusion strategy in enhancing the reliability and accuracy of 3D object detection.