mlsanityData to 14 Oct 2026

Improving Depth Gradient Continuity in Transformers: A Comparative Study on Monocular Depth Estimation with CNN

arXiv 2308.08333 ↗First posted 16 Aug 2023cs.CVcited by 35

Not on the map. Why?

Abstract (arXiv)

Monocular depth estimation is an ongoing challenge in computer vision. Recent progress with Transformer models has demonstrated notable advantages over conventional CNNs in this area. However, there's still a gap in understanding how these models prioritize different regions in 2D images and how these regions affect depth estimation performance. To explore the differences between Transformers and CNNs, we employ a sparse pixel approach to contrastively analyze the distinctions between the two. Our findings suggest that while Transformers excel in handling global context and intricate textures, they lag behind CNNs in preserving depth gradient continuity. To further enhance the performance of Transformer models in monocular depth estimation, we propose the Depth Gradient Refinement (DGR) module that refines depth estimation through high-order differentiation, feature fusion, and recalibration. Additionally, we leverage optimal transport theory, treating depth maps as spatial probability distributions, and employ the optimal transport distance as a loss function to optimize our model. Experimental results demonstrate that models integrated with the plug-and-play Depth Gradient Refinement (DGR) module and the proposed loss function enhance performance without increasing complexity and computational costs on both outdoor KITTI and indoor NYU-Depth-v2 datasets. This research not only offers fresh insights into the distinctions between Transformers and CNNs in depth estimation but also paves the way for novel depth estimation methodologies.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 16 Aug 2023.

Builds on

27 of 37 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  6. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  7. 2021Vision Transformers for Dense Predictioncited by 1,803
  8. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  9. 2020Quantifying Attention Flow in Transformerscited by 775
  10. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
  11. 2020Transformer Interpretability Beyond Attention Visualizationcited by 627
  12. 2020AdaBins: Depth Estimation using Adaptive Binscited by 601
15 more
  1. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 264
  2. 2019Enforcing geometric constraints of virtual normal for depth predictioncited by 234
  3. 2020Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformerscited by 190
  4. 2022NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth Estimationcited by 153
  5. 2018Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundariescited by 145
  6. 2021Transformer-Based Attention Networks for Continuous Pixel-Wise Predictioncited by 125
  7. 2022Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimationcited by 100
  8. 2022Attention Attention Everywhere: Monocular Depth Prediction with Skip Attentioncited by 93
  9. 2022Global-Local Path Networks for Monocular Depth Estimation with Vertical CutDepthcited by 58
  10. 2020CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualizationcited by 57
  11. 2019Visualization of Convolutional Neural Networks for Monocular Depth Estimationcited by 24
  12. 2022Depthformer : Multiscale Vision Transformer For Monocular Depth Estimation With Local Global Information Fusioncited by 21
  13. 2023SwinDepth: Unsupervised Depth Estimation using Monocular Sequences via Swin Transformer and Densely Cascaded Networkcited by 10
  14. 2023DEHRFormer: Real-time Transformer for Depth Estimation and Haze Removal from Varicolored Haze Scenescited by 9
  15. 2023DwinFormer: Dual Window Transformers for End-to-End Monocular Depth Estimationcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 5; 2024: 30.

35 papers.

Sort:

By area:

  1. 2024Retinexmamba: Retinex-based Mamba for Low-light Image Enhancementcited by 59
  2. 2023Deception Detection from Linguistic and Physiological Data Streams Using Bimodal Convolutional Neural Networkscited by 56
  3. 2024Exploring Diverse Methods in Visual Question Answeringcited by 45
  4. 2024Self-Supervised Graph Neural Networks for Enhanced Feature Extraction in Heterogeneous Information Networkscited by 33
  5. 2024Residual-based Language Models are Free Boosters for Biomedical Imagingcited by 24
  6. 2024A Comparative Study on Enhancing Prediction in Social Network Advertisement through Data Augmentationcited by 21
  7. 2024Utilizing Deep Learning to Optimize Software Development Processescited by 15
  8. 2024Efficiency optimization of large-scale language models based on deep learning in natural language processing taskscited by 14
  9. 2023Building Lane-Level Maps from Aerial Imagescited by 13
  10. 2024Application of Multimodal Fusion Deep Learning Model in Disease Recognitioncited by 12
  11. 2024A comparative study of generative adversarial networks for image recognition algorithms based on deep learning and traditional methodscited by 12
  12. 2024Research on Image Recognition Technology Based on Multimodal Deep Learningcited by 11
23 more
  1. 2024Exploration of Attention Mechanism-Enhanced Deep Learning Models in the Mining of Medical Textual Datacited by 11
  2. 2024Enhancing Medical Imaging with GANs Synthesizing Realistic Images from Limited Datacited by 11
  3. 2024Advanced Financial Fraud Detection Using GNN-CL Modelcited by 11
  4. 2024Optimizing YOLOv5s Object Detection through Knowledge Distillation algorithmcited by 11
  5. 2023Top-K Pooling with Patch Contrastive Learning for Weakly-Supervised Semantic Segmentationcited by 10
  6. 2024From Handcrafted Features to LLMs: A Brief Survey for Machine Translation Quality Estimationcited by 10
  7. 2024Exploration of Multi-Scale Image Fusion Systems in Intelligent Medical Image Analysiscited by 10
  8. 2024Algorithm Research of ELMo Word Embedding and Deep Learning Multimodal Transformer in Image Descriptioncited by 10
  9. 2023UniHead: Unifying Multi-Perception for Detection Headscited by 9
  10. 2024Research on Intelligent Aided Diagnosis System of Medical Image Based on Computer Deep Learningcited by 8
  11. 2024Integrating Medical Imaging and Clinical Reports Using Multimodal Deep Learning for Advanced Disease Analysiscited by 8
  12. 2024Robust Graph Neural Networks for Stability Analysis in Dynamic Networkscited by 8
  13. 2024Application of Computer Deep Learning Model in Diagnosis of Pulmonary Nodulescited by 7
  14. 2024Joint-Motion Mutual Learning for Pose Estimation in Videoscited by 7
  15. 2023Adaptive parameter sharing for multi-agent reinforcement learningcited by 6
  16. 2024Research on Early Warning Model of Cardiovascular Disease Based on Computer Deep Learningcited by 6
  17. 2024Research on Disease Prediction Model Construction Based on Computer AI deep Learning Technologycited by 6
  18. 2024Investigation of Customized Medical Decision Algorithms Utilizing Graph Neural Networkscited by 5
  19. 2024Advancing Financial Risk Prediction Through Optimized LSTM Model Performance and Comparative Analysiscited by 5
  20. 2024Research on Optimization of Natural Language Processing Model Based on Multimodal Deep Learningcited by 5
  21. 2024Research on Feature Extraction Data Processing System For MRI of Brain Diseases Based on Computer Deep Learningcited by 5
  22. 2024Coarse-to-fine Alignment Makes Better Speech-image Retrievalcited by 1
  23. 2024Cross-Modal Denoising: A Novel Training Paradigm for Enhancing Speech-Image Retrievalcited by 0