mlsanityData to 7 Oct 2026

Exploring the Mutual Influence between Self-Supervised Single-Frame and Multi-Frame Depth Estimation

arXiv 2304.12685 ↗First posted 25 Apr 2023cs.CV cs.AI eess.IVcited by 0

Not on the map. Why?

Abstract (arXiv)

Although both self-supervised single-frame and multi-frame depth estimation methods only require unlabeled monocular videos for training, the information they leverage varies because single-frame methods mainly rely on appearance-based features while multi-frame methods focus on geometric cues. Considering the complementary information of single-frame and multi-frame methods, some works attempt to leverage single-frame depth to improve multi-frame depth. However, these methods can neither exploit the difference between single-frame depth and multi-frame depth to improve multi-frame depth nor leverage multi-frame depth to optimize single-frame depth models. To fully utilize the mutual influence between single-frame and multi-frame methods, we propose a novel self-supervised training framework. Specifically, we first introduce a pixel-wise adaptive depth sampling module guided by single-frame depth to train the multi-frame model. Then, we leverage the minimum reprojection based distillation loss to transfer the knowledge from the multi-frame depth network to the single-frame network to improve single-frame depth. Finally, we regard the improved single-frame depth as a prior to further boost the performance of multi-frame depth estimation. Experimental results on the KITTI and Cityscapes datasets show that our method outperforms existing approaches in the self-supervised monocular setting.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 45 references that are in the map. First posted on arXiv 25 Apr 2023.

Builds on

36 of 45 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  4. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  5. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  6. 2020Knowledge Distillation: A Surveycited by 1,578
  7. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  8. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  9. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  10. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  11. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  12. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
24 more
  1. 2020AdaBins: Depth Estimation using Adaptive Binscited by 596
  2. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  3. 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
  4. 2017Sparsity Invariant CNNscited by 325
  5. 2019Group-wise Correlation Stereo Networkcited by 235
  6. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
  7. 2020Consistent Video Depth Estimationcited by 220
  8. 2021The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depthcited by 190
  9. 2022NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth Estimationcited by 151
  10. 2020Don't Forget The Past: Recurrent Depth Estimation from Monocular Videocited by 71
  11. 2022Multi-Frame Self-Supervised Depth with Transformerscited by 64
  12. 2021Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistencycited by 58
  13. 2021IterMVS: Iterative Probability Estimation for Efficient Multi-View Stereocited by 56
  14. 2019Refine and Distill: Exploiting Cycle-Inconsistency and Knowledge Distillation for Unsupervised Monocular Depth Estimationcited by 48
  15. 2022Disentangling Object Motion and Occlusion for Unsupervised Multi-frame Monocular Depthcited by 48
  16. 2022RA-Depth: Resolution Adaptive Self-Supervised Monocular Depth Estimationcited by 36
  17. 2021Self-supervised Learning of Depth Inference for Multi-view Stereocited by 28
  18. 2022KD-MVS: Knowledge Distillation Based Self-supervised Learning for Multi-view Stereocited by 18
  19. 2022Self-distilled Feature Aggregation for Self-supervised Monocular Depth Estimationcited by 17
  20. 2022Crafting Monocular Cues and Velocity Guidance for Self-Supervised Multi-Frame Depth Learningcited by 12
  21. 2022JPerceiver: Joint Perception Network for Depth, Pose and Layout Estimation in Driving Scenescited by 11
  22. 2022Semi-Supervised Learning with Mutual Distillation for Monocular Depth Estimationcited by 10
  23. 2022Visual Attention-based Self-supervised Absolute Depth Estimation using Geometric Priors in Autonomous Drivingcited by 8
  24. 2021SUB-Depth: Self-distillation and Uncertainty Boosting Self-supervised Monocular Depth Estimationcited by 7

Built on by

Not cited yet.