mlsanityData to 7 Oct 2026

Fine-grained Semantics-aware Representation Enhancement for Self-supervised Monocular Depth Estimation

arXiv 2108.08829 ↗First posted 19 Aug 2021cs.CVcited by 55

Not on the map. Why?

Abstract (arXiv)

Self-supervised monocular depth estimation has been widely studied, owing to its practical importance and recent promising improvements. However, most works suffer from limited supervision of photometric consistency, especially in weak texture regions and at object boundaries. To overcome this weakness, we propose novel ideas to improve self-supervised monocular depth estimation by leveraging cross-domain information, especially scene semantics. We focus on incorporating implicit semantic knowledge into geometric representation enhancement and suggest two ideas: a metric learning approach that exploits the semantics-guided local geometry to optimize intermediate depth representations and a novel feature fusion module that judiciously utilizes cross-modality between two heterogeneous feature representations. We comprehensively evaluate our methods on the KITTI dataset and demonstrate that our method outperforms state-of-the-art methods. The source code is available at https://github.com/hyBlue/FSRE-Depth.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). Builds on 47 of its 69 references that are in the map. First posted on arXiv 19 Aug 2021.

Builds on

47 of 69 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  6. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  7. 2015Spatial Transformer Networkscited by 1,423
  8. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  9. 2015NetVLAD: CNN architecture for weakly supervised place recognitioncited by 935
  10. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  11. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  12. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
35 more
  1. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 379
  2. 2017Dilated Residual Networkscited by 293
  3. 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
  4. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  5. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
  6. 2019Cross Attention Network for Few-shot Classificationcited by 218
  7. 2014Learning Fine-grained Image Similarity with Deep Rankingcited by 189
  8. 2020On the uncertainty of self-supervised monocular depth estimationcited by 182
  9. 2019Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Camerascited by 170
  10. 2015Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fieldscited by 169
  11. 2020MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learningcited by 157
  12. 2018Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videoscited by 155
  13. 2018DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistencycited by 148
  14. 2020Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidancecited by 148
  15. 2019Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentationcited by 141
  16. 2020Feature-metric Loss for Self-supervised Learning of Depth and Egomotioncited by 140
  17. 2018Improving Semantic Segmentation via Video Propagation and Label Relaxationcited by 134
  18. 2020Semantically-Guided Representation Learning for Self-Supervised Monocular Depthcited by 134
  19. 2019Pixel-Adaptive Convolutional Neural Networkscited by 126
  20. 2020Self-supervised Monocular Trained Depth Estimation using Self-attention and Discrete Disparity Volumecited by 116
  21. 2020HR-Depth: High Resolution Self-Supervised Monocular Depth Estimationcited by 115
  22. 2017Deep Metric Learning with Angular Losscited by 107
  23. 2020Towards Better Generalization: Joint Depth-Pose Learning without PoseNetcited by 85
  24. 2020The Edge of Depth: Explicit Constraints between Segmentation and Depthcited by 72
  25. 2020Don't Forget The Past: Recurrent Depth Estimation from Monocular Videocited by 71
  26. 2020DeFeat-Net: General Monocular Depth via Simultaneous Unsupervised Representation Learningcited by 60
  27. 2021Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistencycited by 58
  28. 2020SOLAR: Second-Order Loss and Attention for Image Retrievalcited by 45
  29. 2020Distilled Semantics for Comprehensive Scene Understanding from Videoscited by 39
  30. 2020Learning Monocular Visual Odometry via Self-Supervised Long-Term Modelingcited by 39
  31. 2018Geometry meets semantics for semi-supervised monocular depth estimationcited by 29
  32. 2019Unsupervised High-Resolution Depth Learning From Videos With Dual Networkscited by 26
  33. 2017Monocular Depth Estimation with Hierarchical Fusion of Dilated CNNs and Soft-Weighted-Sum Inferencecited by 24
  34. 2021Learning Depth via Leveraging Semantics: Self-supervised Monocular Depth Estimation with Both Implicit and Explicit Semantic Guidancecited by 19
  35. 2019Region Deformer Networks for Unsupervised Depth Estimation from Unconstrained Monocular Videoscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 17; 2023: 15; 2024: 17; 2025: 2; 2026: 3.

55 papers.

Sort:

By area:

  1. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  2. 2022BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimationcited by 166
  3. 2022Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimationcited by 100
  4. 2023Learning Semantic-Aware Knowledge Guidance for Low-Light Image Enhancementcited by 57
  5. 2023RoboDepth: Robust Out-of-Distribution Depth Estimation under Corruptionscited by 47
  6. 2022PlaneDepth: Self-supervised Depth Estimation via Orthogonal Planescited by 32
  7. 2023Kick Back & Relax: Learning to Reconstruct the World by Watching SlowTVcited by 21
  8. 2022Towards Scale-Aware, Robust, and Generalizable Unsupervised Monocular Depth Estimation by Integrating IMU Motion Dynamicscited by 20
  9. 2022Self-Supervised Monocular Depth Estimation: Solving the Edge-Fattening Problemcited by 20
  10. 2022Self-Supervised Depth Estimation in Laparoscopic Image using 3D Geometric Consistencycited by 19
  11. 2022Self-distilled Feature Aggregation for Self-supervised Monocular Depth Estimationcited by 17
  12. 2023The RoboDepth Challenge: Methods and Advancements Towards Robust Depth Estimationcited by 13
38 more
  1. 2023From-Ground-To-Objects: Coarse-to-Fine Self-supervised Monocular Depth Estimation of Dynamic Objects with Ground Contact Priorcited by 13
  2. 2023MonoDiffusion: Self-Supervised Monocular Depth Estimation Using Diffusion Modelcited by 12
  3. 2024Know Your Neighbors: Improving Single-View Reconstruction via Spatial Vision-Language Reasoningcited by 12
  4. 2024DCPI-Depth: Explicitly Infusing Dense Correspondence Prior to Unsupervised Monocular Depth Estimationcited by 12
  5. 2024Mono-ViFI: A Unified Learning Framework for Self-supervised Single- and Multi-frame Monocular Depth Estimationcited by 9
  6. 2021THE Benchmark: Transferable Representation Learning for Monocular Height Estimationcited by 8
  7. 2022Visual Attention-based Self-supervised Absolute Depth Estimation using Geometric Priors in Autonomous Drivingcited by 8
  8. 2023Two-in-One Depth: Bridging the Gap Between Monocular and Binocular Self-supervised Depth Estimationcited by 8
  9. 2024ViPOcc: Leveraging Visual Priors from Vision Foundation Models for Single-View 3D Occupancy Predictioncited by 8
  10. 2022MonoDVPS: A Self-Supervised Monocular Depth Estimation Approach to Depth-aware Video Panoptic Segmentationcited by 7
  11. 2023Deeper into Self-Supervised Monocular Indoor Depth Estimationcited by 7
  12. 2024Kick Back & Relax++: Scaling Beyond Ground-Truth Depth with SlowTV & CribsTVcited by 7
  13. 2024SCIPaD: Incorporating Spatial Clues into Unsupervised Pose-Depth Joint Learningcited by 7
  14. 2023Towards Deeply Unified Depth-aware Panoptic Segmentation with Bi-directional Guidance Learningcited by 6
  15. 2024Scene Prior Filtering for Depth Super-Resolutioncited by 5
  16. 2022Monocular Depth Estimation Using Cues Inspired by Biological Vision Systemscited by 4
  17. 2022Globally Consistent Video Depth and Pose Estimation with Efficient Test-Time Trainingcited by 4
  18. 2023Learning Road Scene-level Representations via Semantic Region Predictioncited by 4
  19. 2023Towards Better Data Exploitation in Self-Supervised Monocular Depth Estimationcited by 4
  20. 2024Self-Supervised Monocular Depth Estimation in the Dark: Towards Data Distribution Compensationcited by 4
  21. 2024Improving Domain Generalization in Self-supervised Monocular Depth Estimation via Stabilized Adversarial Trainingcited by 4
  22. 2022Frequency-Aware Self-Supervised Monocular Depth Estimationcited by 3
  23. 2023PPEA-Depth: Progressive Parameter-Efficient Adaptation for Self-Supervised Monocular Depth Estimationcited by 3
  24. 2022Positional Information is All You Need: A Novel Pipeline for Self-Supervised SVDE from Videoscited by 2
  25. 2022On Robust Cross-View Consistency in Self-Supervised Monocular Depth Estimationcited by 2
  26. 2022ROIFormer: Semantic-Aware Region of Interest Transformer for Efficient Self-Supervised Monocular Depth Estimationcited by 2
  27. 2023Dyna-DepthFormer: Multi-frame Transformer for Self-Supervised Depth Estimation in Dynamic Scenescited by 2
  28. 2024FlowDepth: Decoupling Optical Flow for Self-Supervised Monocular Depth Estimationcited by 2
  29. 2026LiftFormer: Lifting and Frame Theory Based Monocular Depth Estimation Using Depth and Edge Oriented Subspace Representationcited by 2
  30. 2023Channel-Aware Distillation Transformer for Depth Estimation on Nano Dronescited by 1
  31. 2023HQDec: Self-Supervised Monocular Depth Estimation Based on a High-Quality Decodercited by 1
  32. 2024A Concise but High-performing Network for Image Guided Depth Completion in Autonomous Drivingcited by 1
  33. 2024Triplet-constraint Transformer with Multi-scale Refinement for Dose Prediction in Radiotherapycited by 1
  34. 2024MetricDepth: Enhancing Monocular Depth Estimation with Deep Metric Learningcited by 1
  35. 2022Detaching and Boosting: Dual Engine for Scale-Invariant Self-Supervised Monocular Depth Estimationcited by 0
  36. 2024DO3D: Self-supervised Learning of Decomposed Object-aware 3D Motion and Depth from Monocular Videoscited by 0
  37. 2024FA-Depth: Toward Fast and Accurate Self-supervised Monocular Depth Estimationcited by 0
  38. 2024Multiple Prior Representation Learning for Self-Supervised Monocular Depth Estimation via Hybrid Transformercited by 0

50 most cited of 55 All 55 (JSON)