# R4Dyn: Exploring Radar for Self-Supervised Monocular Depth Estimation of Dynamic Scenes

arXiv: https://arxiv.org/abs/2108.04814 · HTML: https://mlsanity.com/paper/2108.04814/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 34 references that are in the map.
- First posted on arXiv 10 Aug 2021.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)
- [Scaled-YOLOv4: Scaling Cross Stage Partial Network](https://mlsanity.com/paper/2011.08036/) (2020)
- [Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videos](https://mlsanity.com/paper/1811.06152/) (2018)
- [Semantically-Guided Representation Learning for Self-Supervised Monocular Depth](https://mlsanity.com/paper/2002.12319/) (2020)
- [Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understanding](https://mlsanity.com/paper/1810.06125/) (2018)
- [The Fast Bilateral Solver](https://mlsanity.com/paper/1511.03296/) (2015)
- [Unsupervised Monocular Depth Learning in Dynamic Scenes](https://mlsanity.com/paper/2010.16404/) (2020)
- [A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detection](https://mlsanity.com/paper/2005.07431/) (2020)
- [Self-Supervised Monocular Scene Flow Estimation](https://mlsanity.com/paper/2004.04143/) (2020)
- [Radar-Camera Pixel Depth Association for Depth Completion](https://mlsanity.com/paper/2106.02778/) (2021)
- [Depth Estimation from Monocular Images and Sparse Radar Data](https://mlsanity.com/paper/2010.00058/) (2020)
- [Project to Adapt: Domain Adaptation for Depth Completion from Noisy and Sparse Sensor Data](https://mlsanity.com/paper/2008.01034/) (2020)
- [Every Pixel Counts: Unsupervised Geometry Learning with Holistic 3D Motion Understanding](https://mlsanity.com/paper/1806.10556/) (2018)

## Built on by (most cited first, 30 of 36)

- [Robust Monocular Depth Estimation under Challenging Conditions](https://mlsanity.com/paper/2308.09711/) (2023)
- [3D-VField: Adversarial Augmentation of Point Clouds for Domain Generalization in 3D Object Detection](https://mlsanity.com/paper/2112.04764/) (2021)
- [Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditions](https://mlsanity.com/paper/2407.16698/) (2024)
- [On the Importance of Accurate Geometry Data for Dense 3D Vision Tasks](https://mlsanity.com/paper/2303.14840/) (2023)
- [MonoGraspNet: 6-DoF Grasping with a Single RGB Image](https://mlsanity.com/paper/2209.13036/) (2022)
- [Synthetic-to-Real Self-supervised Robust Depth Estimation via Learning with Motion and Structure Priors](https://mlsanity.com/paper/2503.20211/) (2025)
- [RSA: Resolving Scale Ambiguities in Monocular Depth Estimators through Language Descriptions](https://mlsanity.com/paper/2410.02924/) (2024)
- [Dynamo-Depth: Fixing Unsupervised Depth Estimation for Dynamical Scenes](https://mlsanity.com/paper/2310.18887/) (2023)
- [RadarCam-Depth: Radar-Camera Fusion for Depth Estimation with Learned Metric Scale](https://mlsanity.com/paper/2401.04325/) (2024)
- [Is my Depth Ground-Truth Good Enough? HAMMER -- Highly Accurate Multi-Modal Dataset for DEnse 3D Scene Regression](https://mlsanity.com/paper/2205.04565/) (2022)
- [CertainNet: Sampling-free Uncertainty Estimation for Object Detection](https://mlsanity.com/paper/2110.01604/) (2021)
- [Sparse Beats Dense: Rethinking Supervision in Radar-Camera Depth Completion](https://mlsanity.com/paper/2312.00844/) (2023)
- [Depth Anything at Any Condition](https://mlsanity.com/paper/2507.01634/) (2025)
- [The Third Monocular Depth Estimation Challenge](https://mlsanity.com/paper/2404.16831/) (2024)
- [MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sources](https://mlsanity.com/paper/2601.22054/) (2026)
- [Segmenting Known Objects and Unseen Unknowns without Prior Knowledge](https://mlsanity.com/paper/2209.05407/) (2022)
- [TacoDepth: Towards Efficient Radar-Camera Depth Estimation with One-stage Fusion](https://mlsanity.com/paper/2504.11773/) (2025)
- [The Fourth Monocular Depth Estimation Challenge](https://mlsanity.com/paper/2504.17787/) (2025)
- [Stealing Stable Diffusion Prior for Robust Monocular Depth Estimation](https://mlsanity.com/paper/2403.05056/) (2024)
- [Self-Supervised Radio-Visual Representation Learning for 6G Sensing](https://mlsanity.com/paper/2111.02887/) (2021)
- [Advancing Autonomous Driving: DepthSense with Radar and Spatial Attention](https://mlsanity.com/paper/2109.05265/) (2021)
- [RIDERS: Radar-Infrared Depth Estimation for Robust Sensing](https://mlsanity.com/paper/2402.02067/) (2024)
- [Structure-Aware Radar-Camera Depth Estimation](https://mlsanity.com/paper/2506.05008/) (2025)
- [3D Adversarial Augmentations for Robust Out-of-Domain Predictions](https://mlsanity.com/paper/2308.15479/) (2023)
- [Deep Depth Estimation from Thermal Image: Dataset, Benchmark, and Challenges](https://mlsanity.com/paper/2503.22060/) (2025)
- [Look, Radiate, and Learn: Self-Supervised Localisation via Radio-Visual Correspondence](https://mlsanity.com/paper/2206.06424/) (2022)
- [Bridging Spectral-wise and Multi-spectral Depth Estimation via Geometry-guided Contrastive Learning](https://mlsanity.com/paper/2503.00793/) (2025)
- [RaCalNet: Radar Calibration Network for Sparse-Supervised Metric Depth Estimation](https://mlsanity.com/paper/2506.15560/) (2025)
- [JustDepth: Real-Time Radar-Camera Depth Estimation with Single-Scan LiDAR Supervision](https://mlsanity.com/paper/2607.22172/) (2026)
- [Multi-Modality Driven LoRA for Adverse Condition Depth Estimation](https://mlsanity.com/paper/2412.20162/) (2024)

## Abstract (arXiv)

While self-supervised monocular depth estimation in driving scenarios has achieved comparable performance to supervised approaches, violations of the static world assumption can still lead to erroneous depth predictions of traffic participants, posing a potential safety issue. In this paper, we present R4Dyn, a novel set of techniques to use cost-efficient radar data on top of a self-supervised depth estimation framework. In particular, we show how radar can be used during training as weak supervision signal, as well as an extra input to enhance the estimation robustness at inference time. Since automotive radars are readily available, this allows to collect training data from a variety of existing vehicles. Moreover, by filtering and expanding the signal to make it compatible with learning-based approaches, we address radar inherent issues, such as noise and sparsity. With R4Dyn we are able to overcome a major limitation of self-supervised depth estimation, i.e. the prediction of traffic participants. We substantially improve the estimation on dynamic objects, such as cars by 37% on the challenging nuScenes dataset, hence demonstrating that radar is a valuable additional sensor for monocular depth estimation in autonomous vehicles.