mlsanityData to 14 Oct 2026

Attention meets Geometry: Geometry Guided Spatial-Temporal Attention for Consistent Self-Supervised Monocular Depth Estimation

arXiv 2110.08192 ↗First posted 15 Oct 2021cs.CVcited by 27

Not on the map. Why?

Abstract (arXiv)

Inferring geometrically consistent dense 3D scenes across a tuple of temporally consecutive images remains challenging for self-supervised monocular depth prediction pipelines. This paper explores how the increasingly popular transformer architecture, together with novel regularized loss formulations, can improve depth consistency while preserving accuracy. We propose a spatial attention module that correlates coarse depth predictions to aggregate local geometric information. A novel temporal attention mechanism further processes the local geometric information in a global context across consecutive images. Additionally, we introduce geometric constraints between frames regularized by photometric cycle consistency. By combining our proposed regularization and the novel spatial-temporal-attention module we fully leverage both the geometric and appearance-based consistency across monocular frames. This yields geometrically meaningful attention and improves temporal depth stability and accuracy compared to previous methods.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 47 of its 60 references that are in the map. First posted on arXiv 15 Oct 2021.

Builds on

47 of 60 references.

References · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  5. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  6. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
  7. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 984
  8. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  9. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 806
  10. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  11. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
  12. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
35 more
  1. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 439
  2. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 403
  3. 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
  4. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 342
  5. 2020Exploring Self-attention for Image Recognitioncited by 313
  6. 2017Dilated Residual Networkscited by 294
  7. 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
  8. 2020Consistent Video Depth Estimationcited by 220
  9. 2021The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depthcited by 190
  10. 2020Robust Consistent Video Depth Estimationcited by 188
  11. 2018Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videoscited by 155
  12. 2020Feature-metric Loss for Self-supervised Learning of Depth and Egomotioncited by 140
  13. 2020Self-supervised Monocular Trained Depth Estimation using Self-attention and Discrete Disparity Volumecited by 116
  14. 2017Occlusion Aware Unsupervised Learning of Optical Flowcited by 101
  15. 2020Dual-Resolution Correspondence Networkscited by 90
  16. 2018Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometrycited by 85
  17. 2020Towards Better Generalization: Joint Depth-Pose Learning without PoseNetcited by 85
  18. 2020Guiding Monocular Depth Estimation Using Depth-Attention Volumecited by 82
  19. 2021Monocular Quasi-Dense 3D Object Trackingcited by 81
  20. 2018Joint Monocular 3D Vehicle Detection and Trackingcited by 77
  21. 2016Deep3D: Fully Automatic 2D-to-3D Video Conversion with Deep Convolutional Neural Networkscited by 73
  22. 2020Don't Forget The Past: Recurrent Depth Estimation from Monocular Videocited by 71
  23. 2019Neural RGB->D Sensing: Depth and Uncertainty from a Video Cameracited by 69
  24. 2019Exploiting temporal consistency for real-time video depth estimationcited by 61
  25. 2020MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Cameracited by 59
  26. 2019Visual Odometry Revisited: What Should Be Learnt?cited by 57
  27. 2020Forget About the LiDAR: Self-Supervised Depth Estimators with MED Probability Volumescited by 50
  28. 2019Refine and Distill: Exploiting Cycle-Inconsistency and Knowledge Distillation for Unsupervised Monocular Depth Estimationcited by 49
  29. 2019SteReFo: Efficient Image Refocusing with Stereo Visioncited by 25
  30. 2018Every Pixel Counts: Unsupervised Geometry Learning with Holistic 3D Motion Understandingcited by 24
  31. 2020Monocular Depth Estimation with Self-supervised Instance Adaptationcited by 12
  32. 2018A Deeper Insight into the UnDEMoN: Unsupervised Deep Network for Depth and Ego-Motion Estimationcited by 11
  33. 2020Attentional Separation-and-Aggregation Network for Self-supervised Depth-Pose Learning in Dynamic Scenescited by 11
  34. 2017Camera Pose Filtering with Local Regression Geodesics on the Riemannian Manifold of Dual Quaternionscited by 8
  35. 2020Self-Supervised Attention Learning for Depth and Ego-motion Estimationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 10; 2024: 8; 2025: 1; 2026: 3.

27 papers.

Sort:

By area:

  1. 2022Multi-Frame Self-Supervised Depth with Transformerscited by 64
  2. 2024Cross-View Completion Models are Zero-shot Correspondence Estimatorscited by 30
  3. 2023On the Importance of Accurate Geometry Data for Dense 3D Vision Taskscited by 26
  4. 2023MAMo: Leveraging Memory and Attention for Monocular Video Depth Estimationcited by 24
  5. 2022Is my Depth Ground-Truth Good Enough? HAMMER -- Highly Accurate Multi-Modal Dataset for DEnse 3D Scene Regressioncited by 19
  6. 2024FutureDepth: Learning to Predict the Future Improves Video Depth Estimationcited by 13
  7. 2022Crafting Monocular Cues and Velocity Guidance for Self-Supervised Multi-Frame Depth Learningcited by 12
  8. 2023Manydepth2: Motion-Aware Self-Supervised Monocular Depth Estimation in Dynamic Scenescited by 12
  9. 2024MonoPCC: Photometric-invariant Cycle Constraint for Monocular Depth Estimation of Endoscopic Imagescited by 10
  10. 2024Mono-ViFI: A Unified Learning Framework for Self-supervised Single- and Multi-frame Monocular Depth Estimationcited by 9
  11. 2022GeoRefine: Self-Supervised Online Depth Refinement for Accurate Dense Mappingcited by 8
  12. 2023NVDS+: Towards Efficient and Versatile Neural Stabilizer for Video Depth Estimationcited by 6
15 more
  1. 2024ProDepth: Boosting Self-Supervised Multi-Frame Monocular Depth with Probabilistic Fusioncited by 6
  2. 2023A Study on the Generality of Neural Network Structures for Monocular Depth Estimationcited by 4
  3. 2022CloudAttention: Efficient Multi-Scale Attention Scheme For 3D Point Cloud Learningcited by 2
  4. 2023Dyna-DepthFormer: Multi-frame Transformer for Self-Supervised Depth Estimation in Dynamic Scenescited by 2
  5. 2023Multi-Frame Self-Supervised Depth Estimation with Multi-Scale Feature Fusion in Dynamic Scenescited by 2
  6. 2023ADU-Depth: Attention-based Distillation with Uncertainty Modeling for Depth Estimationcited by 2
  7. 2023HQDec: Self-Supervised Monocular Depth Estimation Based on a High-Quality Decodercited by 1
  8. 2024Geometric Constraints in Deep Learning Frameworks: A Surveycited by 1
  9. 2024MonoDETRNext: Next-Generation Accurate and Efficient Monocular 3D Object Detectorcited by 1
  10. 2024STATIC : Surface Temporal Affine for TIme Consistency in Video Monocular Depth Estimationcited by 1
  11. 2025CylinderDepth: Cylindrical Spatial Attention for Multi-View Consistent Self-Supervised Surround Depth Estimationcited by 1
  12. 2026Interpretable Vision Transformers in Monocular Depth Estimation via SVDAcited by 1
  13. 2023Pose Constraints for Consistent Self-supervised Monocular Depth and Ego-motioncited by 0
  14. 2026Monocular Depth Estimation via Neural Network with Learnable Algebraic Group and Ring Structurescited by 0
  15. 2026CrossDepth: Geometry-Constrained Attention for Generalizable Multi-View Surround Depth Estimationcited by 0