mlsanityData to 7 Oct 2026

Radar-Camera Pixel Depth Association for Depth Completion

arXiv 2106.02778 ↗First posted 5 Jun 2021cs.CVcited by 49

Not on the map. Why?

Abstract (arXiv)

While radar and video data can be readily fused at the detection level, fusing them at the pixel level is potentially more beneficial. This is also more challenging in part due to the sparsity of radar, but also because automotive radar beams are much wider than a typical pixel combined with a large baseline between camera and radar, which results in poor association between radar pixels and color pixel. A consequence is that depth completion methods designed for LiDAR and video fare poorly for radar and video. Here we propose a radar-to-pixel association stage which learns a mapping from radar returns to pixels. This mapping also serves to densify radar returns. Using this as a first stage, followed by a more traditional depth completion method, we are able to achieve image-guided depth completion with radar and video. We demonstrate performance superior to camera and radar alone on the nuScenes dataset. Our source code is available at https://github.com/longyunf/rc-pda.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 35 of its 51 references that are in the map. First posted on arXiv 5 Jun 2021.

Builds on

35 of 51 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  6. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  7. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  8. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  9. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  10. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  11. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  12. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 379
23 more
  1. 2019Panoptic-DeepLabcited by 362
  2. 2017Sparsity Invariant CNNscited by 325
  3. 2019Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challengescited by 313
  4. 2017Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Imagecited by 159
  5. 2020CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detectioncited by 159
  6. 2018Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Cameracited by 140
  7. 2019The Oxford Radar RobotCar Dataset: A Radar Extension to the Oxford RobotCar Datasetcited by 133
  8. 2018DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Imagecited by 129
  9. 2019Self-Supervised Monocular Depth Hintscited by 127
  10. 2020Kinematic 3D Object Detection in Monocular Videocited by 116
  11. 2019Depth Completion from Sparse LiDAR Data with Depth-Normal Constraintscited by 108
  12. 2017SfM-Net: Learning of Structure and Motion from Videocited by 89
  13. 2018Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentationcited by 76
  14. 2020The Edge of Depth: Explicit Constraints between Segmentation and Depthcited by 72
  15. 2020A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detectioncited by 70
  16. 2019Distant Vehicle Detection Using Radar and Visioncited by 44
  17. 2019Depth Coefficients for Depth Completioncited by 41
  18. 2021Depth Completion with Twin Surface Extrapolation at Occlusion Boundariescited by 41
  19. 2017Unsupervised Learning of Geometry with Edge-aware Depth-Normal Consistencycited by 40
  20. 2019Seeing Around Street Corners: Non-Line-of-Sight Detection and Tracking In-the-Wild Using Doppler Radarcited by 40
  21. 2019RRPN: Radar Region Proposal Network for Object Detection in Autonomous Vehiclescited by 33
  22. 2018Geometry meets semantics for semi-supervised monocular depth estimationcited by 29
  23. 2018ConnNet: A Long-Range Relation-Aware Pixel-Connectivity Network for Salient Segmentationcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 6; 2023: 8; 2024: 11; 2025: 12; 2026: 8.

49 papers.

Sort:

By area:

  1. 2023CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perceptioncited by 81
  2. 2022CRAFT: Camera-Radar 3D Object Detection with Spatio-Contextual Fusion Transformercited by 66
  3. 2023Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Reviewcited by 57
  4. 2021R4Dyn: Exploring Radar for Self-Supervised Monocular Depth Estimation of Dynamic Scenescited by 36
  5. 2023Directional Connectivity-based Segmentation of Medical Imagescited by 30
  6. 2023Exploring Radar Data Representations in Autonomous Driving: A Comprehensive Reviewcited by 22
  7. 2022ImmFusion: Robust mmWave-RGB Fusion for 3D Human Body Reconstruction in All Weather Conditionscited by 21
  8. 2024RadarCam-Depth: Radar-Camera Fusion for Depth Estimation with Learned Metric Scalecited by 20
  9. 2024DART: Implicit Doppler Tomography for Radar Novel View Synthesiscited by 19
  10. 2023HVDetFusion: A Simple and Robust Camera-Radar Fusion Frameworkcited by 18
  11. 2021Full-Velocity Radar Returns by Radar-Camera Fusioncited by 16
  12. 2023Sparse Beats Dense: Rethinking Supervision in Radar-Camera Depth Completioncited by 12
37 more
  1. 2024CaFNet: A Confidence-Driven Framework for Radar Camera Depth Estimationcited by 12
  2. 2026MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sourcescited by 10
  3. 2024Radar and Camera Fusion for Object Detection and Tracking: A Comprehensive Surveycited by 9
  4. 2025RICCARDO: Radar Hit Prediction and Convolution for Camera-Radar 3D Object Detectioncited by 9
  5. 2025TacoDepth: Towards Efficient Radar-Camera Depth Estimation with One-stage Fusioncited by 9
  6. 2021Computational Imaging and Artificial Intelligence: The Next Revolution of Mobile Visioncited by 7
  7. 2022RCDPT: Radar-Camera fusion Dense Prediction Transformercited by 7
  8. 2024GET-UP: GEomeTric-aware Depth Estimation with Radar Points UPsamplingcited by 7
  9. 2025ZFusion: An Effective Fuser of Camera and 4D Radar for 3D Object Perception in Autonomous Drivingcited by 6
  10. 2021Advancing Autonomous Driving: DepthSense with Radar and Spatial Attentioncited by 5
  11. 2024RIDERS: Radar-Infrared Depth Estimation for Robust Sensingcited by 5
  12. 2024LiRCDepth: Lightweight Radar-Camera Depth Estimation via Knowledge Distillation and Uncertainty Guidancecited by 5
  13. 2025Radar-Guided Polynomial Fitting for Metric Depth Estimationcited by 5
  14. 2025Structure-Aware Radar-Camera Depth Estimationcited by 5
  15. 2025Deep Depth Estimation from Thermal Image: Dataset, Benchmark, and Challengescited by 4
  16. 2025RC-AutoCalib: An End-to-End Radar-Camera Automatic Calibration Networkcited by 4
  17. 2025Bridging Spectral-wise and Multi-spectral Depth Estimation via Geometry-guided Contrastive Learningcited by 3
  18. 2022How Much Depth Information can Radar Contribute to a Depth Estimation Model?cited by 2
  19. 2022Learning Implicit Functions for Dense 3D Shape Correspondence of Generic Objectscited by 2
  20. 2025RaCalNet: Radar Calibration Network for Sparse-Supervised Metric Depth Estimationcited by 2
  21. 2026JustDepth: Real-Time Radar-Camera Depth Estimation with Single-Scan LiDAR Supervisioncited by 2
  22. 2022Deep learning for radar data exploitation of autonomous vehiclecited by 1
  23. 2023LiDAR Meta Depth Completioncited by 1
  24. 2024VIRUS-NeRF -- Vision, InfraRed and UltraSonic based Neural Radiance Fieldscited by 1
  25. 2024AdaptiveFusion: Adaptive Multi-Modal Multi-View Fusion for 3D Human Body Reconstructioncited by 1
  26. 2025Toward a Low-Cost Perception System in Autonomous Vehicles: A Spectrum Learning Approachcited by 1
  27. 2025TRIDE: A Text-assisted Radar-Image weather-aware fusion network for Depth Estimationcited by 1
  28. 2025XD-RCDepth: Lightweight Radar-Camera Depth Estimation with Explainability-Aligned and Distribution-Aware Distillationcited by 1
  29. 2026Selection, Not Fusion: Radar-Modulated State Space Models for Radar-Camera Depth Estimationcited by 1
  30. 2023DEUX: Active Exploration for Learning Unsupervised Depth Perceptioncited by 0
  31. 2024Depth Estimation fusing Image and Radar Measurements with Uncertain Directionscited by 0
  32. 2024Optical Lens Attack on Monocular Depth Estimation for Autonomous Drivingcited by 0
  33. 2026Instance-Guided Radar Depth Estimation for 3D Object Detectioncited by 0
  34. 2026RadarXFormer: Robust Object Detection via Cross-Dimension Fusion of 4D Radar Spectra and Images for Autonomous Drivingcited by 0
  35. 2026Boosting Robustness for All-Weather Self-Supervised Depth Estimation in Autonomous Drivingcited by 0
  36. 2026RbFT-Net: Rectify-Before-Fuse Temporal Radar Anchors for 4D Radar-Camera Depth Completioncited by 0
  37. 2026GRADE: Single-Frame Generative Radar Depth Estimation Under Visual Degradationcited by 0