mlsanityData to 7 Oct 2026

Camera-Radar Perception for Autonomous Vehicles and ADAS: Concepts, Datasets and Metrics

arXiv 2303.04302 ↗First posted 8 Mar 2023cs.CV cs.AI cs.ROcited by 4

Not on the map. Why?

Abstract (arXiv)

One of the main paths towards the reduction of traffic accidents is the increase in vehicle safety through driver assistance systems or even systems with a complete level of autonomy. In these types of systems, tasks such as obstacle detection and segmentation, especially the Deep Learning-based ones, play a fundamental role in scene understanding for correct and safe navigation. Besides that, the wide variety of sensors in vehicles nowadays provides a rich set of alternatives for improvement in the robustness of perception in challenging situations, such as navigation under lighting and weather adverse conditions. Despite the current focus given to the subject, the literature lacks studies on radar-based and radar-camera fusion-based perception. Hence, this work aims to carry out a study on the current scenario of camera and radar-based perception for ADAS and autonomous vehicles. Concepts and characteristics related to both sensors, as well as to their fusion, are presented. Additionally, we give an overview of the Deep Learning-based detection and segmentation tasks, and the main datasets, metrics, challenges, and open questions in vehicle perception.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 57 of its 122 references that are in the map. First posted on arXiv 8 Mar 2023.

Builds on

57 of 122 references.

References · 57

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2017Mask R-CNNcited by 7,259
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  6. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  8. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  9. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  10. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  11. 2015Fast R-CNNcited by 2,930
  12. 2018YOLOv3: An Incremental Improvementcited by 2,652
38 more
  1. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  2. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  3. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  4. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
  5. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
  6. 2016YOLO9000: Better, Faster, Strongercited by 1,429
  7. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  8. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  9. 2016Playing for Data: Ground Truth from Computer Gamescited by 816
  10. 2018Panoptic Segmentationcited by 796
  11. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  12. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  13. 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
  14. 2019YOLACT: Real-time Instance Segmentationcited by 441
  15. 2019Panoptic-DeepLabcited by 362
  16. 2019Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challengescited by 313
  17. 2017ICNet for Real-Time Semantic Segmentation on High-Resolution Imagescited by 261
  18. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
  19. 2020A2D2: Audi Autonomous Driving Datasetcited by 189
  20. 2018The ApolloScape Open Dataset for Autonomous Driving and its Applicationcited by 188
  21. 2019UPSNet: A Unified Panoptic Segmentation Networkcited by 175
  22. 2017A Review on Deep Learning Techniques Applied to Semantic Segmentationcited by 122
  23. 2019A Survey of Deep Learning-based Object Detectioncited by 106
  24. 2017SegFlow: Joint Learning for Video Object Segmentation and Optical Flowcited by 92
  25. 2020Canadian Adverse Driving Conditions Datasetcited by 86
  26. 2019YOLACT++: Better Real-time Instance Segmentationcited by 84
  27. 2019The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenescited by 80
  28. 2016Semantic Video Segmentation by Gated Recurrent Flow Propagationcited by 76
  29. 2020A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detectioncited by 70
  30. 2019A*3D Dataset: Towards Autonomous Driving in Challenging Environmentscited by 64
  31. 2020CARRADA Dataset: Camera and Automotive Radar with Range-Angle-Doppler Annotationscited by 60
  32. 2018The EuroCity Persons Dataset: A Novel Benchmark for Object Detectioncited by 56
  33. 2019Distant Vehicle Detection Using Radar and Visioncited by 44
  34. 2015Semantic Instance Annotation of Street Scenes by 3D to 2D Label Transfercited by 39
  35. 2020Real-time Fusion Network for RGB-D Semantic Segmentation Incorporating Unexpected Obstacle Detection for Road-driving Imagescited by 36
  36. 2017Choosing Smartly: Adaptive Multimodal Fusion for Object Detection in Changing Environmentscited by 34
  37. 2019RRPN: Radar Region Proposal Network for Object Detection in Autonomous Vehiclescited by 33
  38. 2019Real-Time Panoptic Segmentation from Dense Detectionscited by 31

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 0; 2026: 1.

4 papers.

  1. 2024Deep Learning-Based Robust Multi-Object Tracking via Fusion of mmWave Radar and Camera Sensorscited by 5
  2. 2023A survey on deep learning approaches for data integration in autonomous driving systemcited by 1
  3. 2023Fusing Pseudo Labels with Weak Supervision for Dynamic Traffic Scenarioscited by 0
  4. 2026CLLAP: Contrastive Learning-based LiDAR-Augmented Pretraining for Enhanced Radar-Camera Fusioncited by 0