# Rethinking of Radar's Role: A Camera-Radar Dataset and Systematic Annotator via Coordinate Alignment

arXiv: https://arxiv.org/abs/2105.05207 · HTML: https://mlsanity.com/paper/2105.05207/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 39 references that are in the map.
- First posted on arXiv 11 May 2021.

## Builds on (most cited first, 19 of 19)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges](https://mlsanity.com/paper/1902.07830/) (2019)
- [3D Bounding Box Estimation Using Deep Learning and Geometry](https://mlsanity.com/paper/1612.00496/) (2016)
- [The Oxford Radar RobotCar Dataset: A Radar Extension to the Oxford RobotCar Dataset](https://mlsanity.com/paper/1909.01300/) (2019)
- [RADIATE: A Radar Dataset for Automotive Perception in Bad Weather](https://mlsanity.com/paper/2010.09076/) (2020)
- [CARRADA Dataset: Camera and Automotive Radar with Range-Angle-Doppler Annotations](https://mlsanity.com/paper/2005.01456/) (2020)
- [RODNet: A Real-Time Radar Object Detection Network Cross-Supervised by Camera-Radar Fused Object 3D Localization](https://mlsanity.com/paper/2102.05150/) (2021)
- [CNN based Road User Detection using the 3D Radar Cube](https://mlsanity.com/paper/2004.12165/) (2020)
- [Exploit the Connectivity: Multi-Object Tracking with TrackletNet](https://mlsanity.com/paper/1811.07258/) (2018)
- [Probabilistic Oriented Object Detection in Automotive Radar](https://mlsanity.com/paper/2004.05310/) (2020)
- [Reconstructing Vechicles from a Single Image: Shape Priors for Road Scene Understanding](https://mlsanity.com/paper/1609.09468/) (2016)
- [The Earth ain't Flat: Monocular Reconstruction of Vehicles on Steep and Graded Roads from a Moving Camera](https://mlsanity.com/paper/1803.02057/) (2018)
- [Traffic-Aware Multi-Camera Tracking of Vehicles Based on ReID and Camera Link Model](https://mlsanity.com/paper/2008.09785/) (2020)

## Built on by (most cited first, 23 of 23)

- [K-Radar: 4D Radar Object Detection for Autonomous Driving in Various Weather Conditions](https://mlsanity.com/paper/2206.08171/) (2022)
- [Raw High-Definition Radar for Multi-Task Learning](https://mlsanity.com/paper/2112.10646/) (2021)
- [Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2304.10410/) (2023)
- [mmBody Benchmark: 3D Body Reconstruction Dataset and Analysis for Millimeter Wave Radar](https://mlsanity.com/paper/2209.05070/) (2022)
- [Exploiting Temporal Relations on Radar Perception for Autonomous Driving](https://mlsanity.com/paper/2204.01184/) (2022)
- [Exploring Radar Data Representations in Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2312.04861/) (2023)
- [ImmFusion: Robust mmWave-RGB Fusion for 3D Human Body Reconstruction in All Weather Conditions](https://mlsanity.com/paper/2210.01346/) (2022)
- [DPFT: Dual Perspective Fusion Transformer for Camera-Radar-based Object Detection](https://mlsanity.com/paper/2404.03015/) (2024)
- [ADCNet: Learning from Raw Radar Data via Distillation](https://mlsanity.com/paper/2303.11420/) (2023)
- [Contrastive Learning for Automotive mmWave Radar Detection Points Based Instance Segmentation](https://mlsanity.com/paper/2203.06553/) (2022)
- [A recurrent CNN for online object detection on raw radar frames](https://mlsanity.com/paper/2212.11172/) (2022)
- [Bootstrapping Autonomous Driving Radars with Self-Supervised Learning](https://mlsanity.com/paper/2312.04519/) (2023)
- [Radar and Camera Fusion for Object Detection and Tracking: A Comprehensive Survey](https://mlsanity.com/paper/2410.19872/) (2024)
- [HyperSense: Hyperdimensional Intelligent Sensing for Energy-Efficient Sparse Data Processing](https://mlsanity.com/paper/2401.10267/) (2024)
- [Vision meets mmWave Radar: 3D Object Perception Benchmark for Autonomous Driving](https://mlsanity.com/paper/2311.10261/) (2023)
- [4D Radar Semantic Segmentation of People in Field Conditions Using Temporal Multi-View Networks](https://mlsanity.com/paper/2404.05307/) (2024)
- [DeepSense-V2V: A Vehicle-to-Vehicle Multi-Modal Sensing, Localization, and Communications Dataset](https://mlsanity.com/paper/2406.17908/) (2024)
- [Exploring Domain Shift on Radar-Based 3D Object Detection Amidst Diverse Environmental Conditions](https://mlsanity.com/paper/2408.06772/) (2024)
- [Rad-R: A Raw-ADC Radar Dataset and Capture-Invariant SSM for Hardware-Fault Diagnosis](https://mlsanity.com/paper/2608.30896/) (2026)
- [Radar-Camera Fused Multi-Object Tracking: Online Calibration and Common Feature](https://mlsanity.com/paper/2510.20794/) (2025)
- [Synthetic FMCW Radar Range Azimuth Maps Augmentation with Generative Diffusion Model](https://mlsanity.com/paper/2601.06228/) (2026)
- [AdaRadar: Rate Adaptive Spectral Compression for Radar-based Perception](https://mlsanity.com/paper/2603.17979/) (2026)
- [Revisiting Radar Perception With Spectral Point Clouds](https://mlsanity.com/paper/2604.08282/) (2026)

## Abstract (arXiv)

Radar has long been a common sensor on autonomous vehicles for obstacle ranging and speed estimation. However, as a robust sensor to all-weather conditions, radar's capability has not been well-exploited, compared with camera or LiDAR. Instead of just serving as a supplementary sensor, radar's rich information hidden in the radio frequencies can potentially provide useful clues to achieve more complicated tasks, like object classification and detection. In this paper, we propose a new dataset, named CRUW, with a systematic annotator and performance evaluation system to address the radar object detection (ROD) task, which aims to classify and localize the objects in 3D purely from radar's radio frequency (RF) images. To the best of our knowledge, CRUW is the first public large-scale dataset with a systematic annotation and evaluation system, which involves camera RGB images and radar RF images, collected in various driving scenarios.