# RadarCam-Depth: Radar-Camera Fusion for Depth Estimation with Learned Metric Scale

arXiv: https://arxiv.org/abs/2401.04325 · HTML: https://mlsanity.com/paper/2401.04325/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 38 references that are in the map.
- First posted on arXiv 9 Jan 2024.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video](https://mlsanity.com/paper/1908.10553/) (2019)
- [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](https://mlsanity.com/paper/2104.00681/) (2021)
- [DeepV2D: Video to Depth with Differentiable Structure from Motion](https://mlsanity.com/paper/1812.04605/) (2018)
- [On the uncertainty of self-supervised monocular depth estimation](https://mlsanity.com/paper/2005.06209/) (2020)
- [MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimation](https://mlsanity.com/paper/2307.14460/) (2023)
- [Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image](https://mlsanity.com/paper/1709.07492/) (2017)
- [Towards Zero-Shot Scale-Aware Monocular Depth Estimation](https://mlsanity.com/paper/2306.17253/) (2023)
- [Unsupervised Depth Completion from Visual Inertial Odometry](https://mlsanity.com/paper/1905.08616/) (2019)
- [Towards Better Generalization: Joint Depth-Pose Learning without PoseNet](https://mlsanity.com/paper/2004.01314/) (2020)
- [Sparse and noisy LiDAR completion with RGB guidance and uncertainty](https://mlsanity.com/paper/1902.05356/) (2019)
- [Unsupervised Depth Completion with Calibrated Backprojection Layers](https://mlsanity.com/paper/2108.10531/) (2021)
- [Dense Depth Posterior (DDP) from Single Image and Sparse Range](https://mlsanity.com/paper/1901.10034/) (2019)
- [Radar-Camera Pixel Depth Association for Depth Completion](https://mlsanity.com/paper/2106.02778/) (2021)
- [Depth Estimation from Monocular Images and Sparse Radar Data](https://mlsanity.com/paper/2010.00058/) (2020)
- [R4Dyn: Exploring Radar for Self-Supervised Monocular Depth Estimation of Dynamic Scenes](https://mlsanity.com/paper/2108.04814/) (2021)
- [Depth Estimation from Monocular Images and Sparse radar using Deep Ordinal Regression Network](https://mlsanity.com/paper/2107.07596/) (2021)
- [Improving Semi-Supervised and Domain-Adaptive Semantic Segmentation with Self-Supervised Depth Estimation](https://mlsanity.com/paper/2108.12545/) (2021)
- [Video Depth Estimation by Fusing Flow-to-Depth Proposals](https://mlsanity.com/paper/1912.12874/) (2019)
- [CodeVIO: Visual-Inertial Odometry with Learned Optimizable Dense Depth](https://mlsanity.com/paper/2012.10133/) (2020)
- [Monocular Visual-Inertial Depth Estimation](https://mlsanity.com/paper/2303.12134/) (2023)
- [Full-Velocity Radar Returns by Radar-Camera Fusion](https://mlsanity.com/paper/2108.10637/) (2021)
- [Deep Depth Estimation from Visual-Inertial SLAM](https://mlsanity.com/paper/2008.00092/) (2020)

## Built on by (most cited first, 20 of 20)

- [OMNI-DC: Highly Robust Depth Completion with Multiresolution Depth Integration](https://mlsanity.com/paper/2411.19278/) (2024)
- [Diffusion Knows Transparency: Repurposing Video Diffusion for Transparent Object Depth and Normal Estimation](https://mlsanity.com/paper/2512.23705/) (2025)
- [MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sources](https://mlsanity.com/paper/2601.22054/) (2026)
- [Radar and Camera Fusion for Object Detection and Tracking: A Comprehensive Survey](https://mlsanity.com/paper/2410.19872/) (2024)
- [TacoDepth: Towards Efficient Radar-Camera Depth Estimation with One-stage Fusion](https://mlsanity.com/paper/2504.11773/) (2025)
- [RIDERS: Radar-Infrared Depth Estimation for Robust Sensing](https://mlsanity.com/paper/2402.02067/) (2024)
- [Radar-Guided Polynomial Fitting for Metric Depth Estimation](https://mlsanity.com/paper/2503.17182/) (2025)
- [Structure-Aware Radar-Camera Depth Estimation](https://mlsanity.com/paper/2506.05008/) (2025)
- [TanDepth: Leveraging Global DEMs for Metric Monocular Depth Estimation in UAVs](https://mlsanity.com/paper/2409.05142/) (2024)
- [A Survey of Deep Learning Based Radar and Vision Fusion for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/2406.00714/) (2024)
- [RaCalNet: Radar Calibration Network for Sparse-Supervised Metric Depth Estimation](https://mlsanity.com/paper/2506.15560/) (2025)
- [JustDepth: Real-Time Radar-Camera Depth Estimation with Single-Scan LiDAR Supervision](https://mlsanity.com/paper/2607.22172/) (2026)
- [Toward a Low-Cost Perception System in Autonomous Vehicles: A Spectrum Learning Approach](https://mlsanity.com/paper/2502.01940/) (2025)
- [TRIDE: A Text-assisted Radar-Image weather-aware fusion network for Depth Estimation](https://mlsanity.com/paper/2508.08038/) (2025)
- [XD-RCDepth: Lightweight Radar-Camera Depth Estimation with Explainability-Aligned and Distribution-Aware Distillation](https://mlsanity.com/paper/2510.13565/) (2025)
- [Selection, Not Fusion: Radar-Modulated State Space Models for Radar-Camera Depth Estimation](https://mlsanity.com/paper/2605.11840/) (2026)
- [AVS-Net: Audio-Visual Scale Net for Self-supervised Monocular Metric Depth Estimation](https://mlsanity.com/paper/2412.01637/) (2024)
- [SwiftNDC: Fast Neural Depth Correction for High-Fidelity 3D Reconstruction](https://mlsanity.com/paper/2602.22565/) (2026)
- [RbFT-Net: Rectify-Before-Fuse Temporal Radar Anchors for 4D Radar-Camera Depth Completion](https://mlsanity.com/paper/2608.13102/) (2026)
- [GRADE: Single-Frame Generative Radar Depth Estimation Under Visual Degradation](https://mlsanity.com/paper/2609.10756/) (2026)

## Abstract (arXiv)

We present a novel approach for metric dense depth estimation based on the fusion of a single-view image and a sparse, noisy Radar point cloud. The direct fusion of heterogeneous Radar and image data, or their encodings, tends to yield dense depth maps with significant artifacts, blurred boundaries, and suboptimal accuracy. To circumvent this issue, we learn to augment versatile and robust monocular depth prediction with the dense metric scale induced from sparse and noisy Radar data. We propose a Radar-Camera framework for highly accurate and fine-detailed dense depth estimation with four stages, including monocular depth prediction, global scale alignment of monocular depth with sparse Radar points, quasi-dense scale estimation through learning the association between Radar points and image patches, and local scale refinement of dense depth using a scale map learner. Our proposed method significantly outperforms the state-of-the-art Radar-Camera depth estimation methods by reducing the mean absolute error (MAE) of depth estimation by 25.6% and 40.2% on the challenging nuScenes dataset and our self-collected ZJU-4DRadarCam dataset, respectively. Our code and dataset will be released at urlhttps://github.com/MMOCKING/RadarCam-Depth.