mlsanityData to 7 Oct 2026

On Robust Cross-View Consistency in Self-Supervised Monocular Depth Estimation

arXiv 2209.08747 ↗First posted 19 Sep 2022cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Remarkable progress has been made in self-supervised monocular depth estimation (SS-MDE) by exploring cross-view consistency, e.g., photometric consistency and 3D point cloud consistency. However, they are very vulnerable to illumination variance, occlusions, texture-less regions, as well as moving objects, making them not robust enough to deal with various scenes. To address this challenge, we study two kinds of robust cross-view consistency in this paper. Firstly, the spatial offset field between adjacent frames is obtained by reconstructing the reference frame from its neighbors via deformable alignment, which is used to align the temporal depth features via a Depth Feature Alignment (DFA) loss. Secondly, the 3D point clouds of each reference frame and its nearby frames are calculated and transformed into voxel space, where the point density in each voxel is calculated and aligned via a Voxel Density Alignment (VDA) loss. In this way, we exploit the temporal coherence in both depth feature space and 3D voxel space for SS-MDE, shifting the "point-to-point" alignment paradigm to the "region-to-region" one. Compared with the photometric consistency loss as well as the rigid point cloud alignment loss, the proposed DFA and VDA losses are more robust owing to the strong representation power of deep features as well as the high tolerance of voxel density to the aforementioned challenges. Experimental results on several outdoor benchmarks show that our method outperforms current state-of-the-art techniques. Extensive ablation study and analysis validate the effectiveness of the proposed losses, especially in challenging scenes. The code and models are available at https://github.com/sunnyHelen/RCVC-depth.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 43 of its 63 references that are in the map. First posted on arXiv 19 Sep 2022.

Builds on

43 of 63 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  4. 2017Deformable Convolutional Networkscited by 1,481
  5. 2015Spatial Transformer Networkscited by 1,423
  6. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  7. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  8. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  9. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  10. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  11. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  12. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
31 more
  1. 2020Virtual KITTI 2cited by 345
  2. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 341
  3. 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
  4. 2018TDAN: Temporally Deformable Alignment Network for Video Super-Resolutioncited by 225
  5. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 225
  6. 2020Consistent Video Depth Estimationcited by 220
  7. 2018Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraintscited by 179
  8. 2019Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Camerascited by 170
  9. 2018Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstructioncited by 159
  10. 2018Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videoscited by 155
  11. 2017Learning Depth from Monocular Videos using Direct Methodscited by 154
  12. 2018DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistencycited by 148
  13. 2020Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidancecited by 148
  14. 2020Feature-metric Loss for Self-supervised Learning of Depth and Egomotioncited by 140
  15. 2022MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformercited by 136
  16. 2019Self-supervised Learning with Geometric Constraints in Monocular Video: Connecting Flow, Depth, and Cameracited by 130
  17. 2020Self-supervised Monocular Trained Depth Estimation using Self-attention and Discrete Disparity Volumecited by 116
  18. 2020HR-Depth: High Resolution Self-Supervised Monocular Depth Estimationcited by 115
  19. 2020Towards Better Generalization: Joint Depth-Pose Learning without PoseNetcited by 85
  20. 2020Understanding Deformable Alignment in Video Super-Resolutioncited by 78
  21. 2021Self-Supervised Monocular Depth Estimation with Internal Feature Fusioncited by 67
  22. 2021Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimationcited by 63
  23. 2021Fine-grained Semantics-aware Representation Enhancement for Self-supervised Monocular Depth Estimationcited by 55
  24. 2018LEGO: Learning Edge with Geometry all at Once by Watching Videoscited by 47
  25. 2019Unsupervised High-Resolution Depth Learning From Videos With Dual Networkscited by 26
  26. 2022Deep Digging into the Generalization of Self-Supervised Monocular Depth Estimationcited by 24
  27. 2022Towards Scale-Aware, Robust, and Generalizable Unsupervised Monocular Depth Estimation by Integrating IMU Motion Dynamicscited by 20
  28. 2021Learning Depth via Leveraging Semantics: Self-supervised Monocular Depth Estimation with Both Implicit and Explicit Semantic Guidancecited by 19
  29. 20213D Hierarchical Refinement and Augmentation for Unsupervised Learning of Depth and Pose from Monocular Videocited by 11
  30. 2022JPerceiver: Joint Perception Network for Depth, Pose and Layout Estimation in Driving Scenescited by 11
  31. 2022Towards Scale Consistent Monocular Visual Odometry by Learning from the Virtual Worldcited by 8

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024GVGEN: Text-to-3D Generation with Volumetric Representationcited by 51
  2. 2024Geometric Constraints in Deep Learning Frameworks: A Surveycited by 1