mlsanityData to 7 Oct 2026

Monocular Depth Estimation through Virtual-world Supervision and Real-world SfM Self-Supervision

arXiv 2103.12209 ↗First posted 22 Mar 2021cs.CVcited by 24

Not on the map. Why?

Abstract (arXiv)

Depth information is essential for on-board perception in autonomous driving and driver assistance. Monocular depth estimation (MDE) is very appealing since it allows for appearance and depth being on direct pixelwise correspondence without further calibration. Best MDE models are based on Convolutional Neural Networks (CNNs) trained in a supervised manner, i.e., assuming pixelwise ground truth (GT). Usually, this GT is acquired at training time through a calibrated multi-modal suite of sensors. However, also using only a monocular system at training time is cheaper and more scalable. This is possible by relying on structure-from-motion (SfM) principles to generate self-supervision. Nevertheless, problems of camouflaged objects, visibility changes, static-camera intervals, textureless areas, and scale ambiguity, diminish the usefulness of such self-supervision. In this paper, we perform monocular depth estimation by virtual-world supervision (MonoDEVS) and real-world SfM self-supervision. We compensate the SfM self-supervision limitations by leveraging virtual-world images with accurate semantic and depth supervision and addressing the virtual-to-real domain gap. Our MonoDEVSNet outperforms previous MDE CNNs trained on monocular and even stereo sequences.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 48 of its 70 references that are in the map. First posted on arXiv 22 Mar 2021.

Builds on

48 of 70 references.

References · 48

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  4. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  5. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  6. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  7. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  8. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  9. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  10. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  11. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  12. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
36 more
  1. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  2. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  3. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  4. 2018Deep Visual Domain Adaptation: A Surveycited by 634
  5. 2020AdaBins: Depth Estimation using Adaptive Binscited by 596
  6. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 517
  7. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  8. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 402
  9. 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
  10. 2018A Survey of Unsupervised Deep Domain Adaptationcited by 361
  11. 2020Virtual KITTI 2cited by 345
  12. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 341
  13. 2016Virtual Worlds as Proxy for Multi-Object Tracking Analysiscited by 340
  14. 2019Enforcing geometric constraints of virtual normal for depth predictioncited by 233
  15. 2015Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fieldscited by 169
  16. 2017Playing for Benchmarkscited by 169
  17. 2020Hierarchical Neural Architecture Search for Deep Stereo Matchingcited by 166
  18. 2017A Survey of Structure from Motioncited by 157
  19. 2020Semantically-Guided Representation Learning for Self-Supervised Monocular Depthcited by 134
  20. 2017Semi-Supervised Deep Learning for Monocular Depth Map Predictioncited by 109
  21. 2020Convolutional Networks with Dense Connectivitycited by 98
  22. 2016Estimating Depth from Monocular Images as Classification Using Deep Fully Convolutional Residual Networkscited by 91
  23. 2020Towards Better Generalization: Joint Depth-Pose Learning without PoseNetcited by 85
  24. 2018Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimationcited by 78
  25. 2018T2Net: Synthetic-to-Realistic Translation for Solving Single-Image Depth Estimation Taskscited by 70
  26. 2018SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimationcited by 64
  27. 2019Geometry-Aware Symmetric Domain Adaptation for Monocular Depth Estimationcited by 61
  28. 2018AdaDepth: Unsupervised Content Congruent Adaptation for Depth Estimationcited by 54
  29. 2016Joint Semantic Segmentation and Depth Estimation with Deep Convolutional Networkscited by 36
  30. 2019Robust Semi-Supervised Monocular Depth Estimation with Reprojected Distancescited by 27
  31. 2018Learning Depth from Single Images with Deep Neural Network Embedding Focal Lengthcited by 25
  32. 2020SharinGAN: Combining Synthetic and Real Data for Unsupervised Geometry Estimationcited by 25
  33. 2020On Deep Learning Techniques to Boost Monocular Depth Estimation for Autonomous Navigationcited by 17
  34. 2020S3Net: Semantic-Aware Self-supervised Depth Estimation with Monocular Videos and Synthetic Datacited by 12
  35. 2017Analyzing Modular CNN Architectures for Joint Depth Prediction and Semantic Segmentationcited by 8
  36. 2018Monocular Depth Estimation by Learning from Heterogeneous Datasetscited by 3

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 6; 2023: 9; 2024: 4; 2025: 3.

24 papers.

Sort:

By area:

  1. 2023Towards Zero-Shot Scale-Aware Monocular Depth Estimationcited by 100
  2. 2022Multi-Frame Self-Supervised Depth with Transformerscited by 64
  3. 2023Self-supervised Monocular Depth Estimation: Let's Talk About The Weathercited by 33
  4. 2022Learning Optical Flow, Depth, and Scene Flow without Real-World Labelscited by 31
  5. 2021Unsupervised Monocular Depth Estimation in Highly Complex Environmentscited by 28
  6. 2023The Second Monocular Depth Estimation Challengecited by 12
  7. 2023SRFNet: Monocular Depth Estimation with Fine-grained Structure via Spatial Reliability-oriented Fusion of Frames and Eventscited by 11
  8. 2023Self-Supervised Monocular Depth Estimation with Self-Reference Distillation and Disparity Offset Refinementcited by 10
  9. 2022Mind The Edge: Refining Depth Edges in Sparsely-Supervised Monocular Depth Estimationcited by 8
  10. 2024Synthetic Data Generation Framework, Dataset, and Efficient Deep Model for Pedestrian Intention Predictioncited by 5
  11. 2024Improving Domain Generalization in Self-supervised Monocular Depth Estimation via Stabilized Adversarial Trainingcited by 4
  12. 2021Co-training for Deep Object Detection: Comparing Single-modal and Multi-modal Approachescited by 2
12 more
  1. 2022ROIFormer: Semantic-Aware Region of Interest Transformer for Efficient Self-Supervised Monocular Depth Estimationcited by 2
  2. 2023DeepIPCv2: LiDAR-powered Robust Environmental Perception and Navigational Control for Autonomous Vehiclecited by 2
  3. 2023GenDepth: Generalizing Monocular Depth Estimation for Arbitrary Camera Parameters via Ground Plane Embeddingcited by 2
  4. 2023On the Metrics for Evaluating Monocular Depth Estimationcited by 1
  5. 2023Depth-Relative Self Attention for Monocular Depth Estimationcited by 1
  6. 2024Virtually Enriched NYU Depth V2 Dataset for Monocular Depth Estimation: Do We Need Artificial Augmentation?cited by 1
  7. 2025An Online Adaptation Method for Robust Depth Estimation and Visual Odometry in the Open Worldcited by 1
  8. 2025UniCT Depth: Event-Image Fusion Based Monocular Depth Estimation with Convolution-Compensated ViT Dual SA Blockcited by 1
  9. 2025Guided Diffusion-based Generation of Adversarial Objects for Real-World Monocular Depth Estimation Attackscited by 1
  10. 2022Deep Camera Pose Regression Using Pseudo-LiDARcited by 0
  11. 2022Multilingual Communication System with Deaf Individuals Utilizing Natural and Visual Languagescited by 0
  12. 2024Back2Color: Domain-Adaptive Synthetic-to-Real Monocular Depth Estimation for Dynamic Traffic Scenescited by 0