Boosting Instance Awareness via Cross-View Correlation with 4D Radar and Camera for 3D Object Detection
Not on the map. Why?
Abstract (arXiv)
4D millimeter-wave radar has emerged as a promising sensing modality for autonomous driving due to its robustness and affordability. However, its sparse and weak geometric cues make reliable instance activation difficult, limiting the effectiveness of existing radar-camera fusion paradigms. BEV-level fusion offers global scene understanding but suffers from weak instance focus, while perspective-level fusion captures instance details but lacks holistic context. To address these limitations, we propose SIFormer, a scene-instance aware transformer for 3D object detection using 4D radar and camera. SIFormer first suppresses background noise during view transformation through segmentation- and depth-guided localization. It then introduces a cross-view activation mechanism that injects 2D instance cues into BEV space, enabling reliable instance awareness under weak radar geometry. Finally, a transformer-based fusion module aggregates complementary image semantics and radar geometry for robust perception. As a result, with the aim of enhancing instance awareness, SIFormer bridges the gap between the two paradigms, combining their complementary strengths to address inherent sparse nature of radar and improve detection accuracy. Experiments demonstrate that SIFormer achieves state-of-the-art performance on View-of-Delft, TJ4DRadSet and NuScenes datasets. Source code is available at github.com/shawnnnkb/SIFormer.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 39 of its 57 references that are in the map. First posted on arXiv 24 Feb 2026.
Builds on
39 of 57 references.
References · 39
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
- 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
- 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
- 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
- 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
- 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
- 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
- 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
- 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 464
- 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
- 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
27 more
- 2019Cascade R-CNN: High Quality Object Detection and Instance Segmentationcited by 428
- 2022Neighborhood Attention Transformercited by 235
- 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
- 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 201
- 2022FUTR3D: A Unified Sensor Fusion Framework for 3D Detectioncited by 194
- 2020CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detectioncited by 159
- 2021ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detectioncited by 157
- 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
- 2022Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?cited by 128
- 2022DeepInteraction: 3D Object Detection via Modality Interactioncited by 125
- 2023FB-BEV: BEV Representation from Forward-Backward View Transformationscited by 98
- 2024RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detectioncited by 82
- 2021Perception and Sensing for Autonomous Vehicles Under Adverse Weather Conditions: A Surveycited by 81
- 2023CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perceptioncited by 81
- 2022TJ4DRadSet: A 4D Radar Dataset for Autonomous Drivingcited by 74
- 2023UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representationcited by 74
- 2023Far3D: Expanding the Horizon for Surround-view 3D Object Detectioncited by 70
- 2022CRAFT: Camera-Radar 3D Object Detection with Spatio-Contextual Fusion Transformercited by 66
- 2023LXL: LiDAR Excluded Lean 3D Object Detection with 4D Imaging Radar and Camera Fusioncited by 57
- 2024IS-Fusion: Instance-Scene Collaborative Fusion for Multimodal 3D Object Detectioncited by 57
- 2023SMURF: Spatial Multi-Representation Fusion for 3D Object Detection with 4D Imaging Radarcited by 48
- 2023X3KD: Knowledge Distillation Across Modalities, Tasks and Stages for Multi-Camera 3D Object Detectioncited by 28
- 2023RCM-Fusion: Radar-Camera Multi-Level Fusion for 3D Object Detectioncited by 23
- 2025Doracamom: Joint 3D Detection and Occupancy Prediction with Multi-view 4D Radars and Cameras for Omnidirectional Perceptioncited by 21
- 2025RaGS: Unleashing 3D Gaussian Splatting from 4D Radar and Monocular Cues for 3D Object Detectioncited by 9
- 2024SparseFusion: Efficient Sparse Multi-Modal Fusion Framework for Long-Range 3D Perceptioncited by 8
- 2025Wavelet-based Multi-View Fusion of 4D Radar Tensor and Camera for Robust 3D Object Detectioncited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.