mlsanityData to 7 Oct 2026

Transformer-Based Attention Networks for Continuous Pixel-Wise Prediction

arXiv 2103.12091 ↗First posted 22 Mar 2021cs.CVcited by 125

Not on the map. Why?

Abstract (arXiv)

While convolutional neural networks have shown a tremendous impact on various computer vision tasks, they generally demonstrate limitations in explicitly modeling long-range dependencies due to the intrinsic locality of the convolution operation. Initially designed for natural language processing tasks, Transformers have emerged as alternative architectures with innate global self-attention mechanisms to capture long-range dependencies. In this paper, we propose TransDepth, an architecture that benefits from both convolutional neural networks and transformers. To avoid the network losing its ability to capture local-level details due to the adoption of transformers, we propose a novel decoder that employs attention mechanisms based on gates. Notably, this is the first paper that applies transformers to pixel-wise prediction problems involving continuous labels (i.e., monocular depth prediction and surface normal estimation). Extensive experiments demonstrate that the proposed TransDepth achieves state-of-the-art performance on three challenging datasets. Our code is available at: https://github.com/ygjwd12345/TransDepth.

Key facts. Cited by 125 papers in the map (data through 7 Oct 2026). Builds on 55 of its 73 references that are in the map. First posted on arXiv 22 Mar 2021.

Builds on

55 of 73 references.

References · 55

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  6. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  7. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  8. 2016Pyramid Scene Parsing Networkcited by 2,472
  9. 2017Non-local Neural Networkscited by 2,290
  10. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  11. 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,552
  12. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
38 more
  1. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  2. 2018Dual Attention Network for Scene Segmentationcited by 1,001
  3. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  4. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
  5. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  6. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  7. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 402
  8. 2015Attention-Based Models for Speech Recognitioncited by 400
  9. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 379
  10. 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
  11. 2018Context Encoding for Semantic Segmentationcited by 289
  12. 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
  13. 2019Enforcing geometric constraints of virtual normal for depth predictioncited by 233
  14. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  15. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
  16. 2019Local Relation Networks for Image Recognitioncited by 210
  17. 2017Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Networkcited by 208
  18. 2015Attention to Scale: Scale-aware Semantic Image Segmentationcited by 169
  19. 2020Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidancecited by 148
  20. 2020Feature-metric Loss for Self-supervised Learning of Depth and Egomotioncited by 140
  21. 2020Semantically-Guided Representation Learning for Self-Supervised Monocular Depthcited by 134
  22. 2020Self-supervised Monocular Trained Depth Estimation using Self-attention and Discrete Disparity Volumecited by 116
  23. 2020End-to-end Lane Shape Prediction with Transformerscited by 111
  24. 2014Deep Convolutional Neural Fields for Depth Estimation from a Single Imagecited by 106
  25. 2019Multi-Channel Attention Selection GAN with Cascaded Semantic Guidance for Cross-View Image Translationcited by 97
  26. 2020Guiding Monocular Depth Estimation Using Depth-Attention Volumecited by 82
  27. 2020XingGAN for Person Image Generationcited by 81
  28. 2018Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimationcited by 78
  29. 2017Multi-Scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimationcited by 70
  30. 2014The Application of Two-level Attention Models in Deep Convolutional Neural Network for Fine-grained Image Classificationcited by 67
  31. 2020DeFeat-Net: General Monocular Depth via Simultaneous Unsupervised Representation Learningcited by 60
  32. 2020Forget About the LiDAR: Self-Supervised Depth Estimators with MED Probability Volumescited by 50
  33. 2020Pseudo RGB-D for Self-Improving Monocular SLAM and Depth Predictioncited by 41
  34. 2020Dual Attention GANs for Semantic Image Synthesiscited by 41
  35. 2019Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translationcited by 39
  36. 2016Marr Revisited: 2D-3D Alignment via Surface Normal Predictioncited by 34
  37. 2020Surface Normal Estimation of Tilted Images via Spatial Rectifiercited by 34
  38. 2019FrameNet: Learning Local Canonical Frames of 3D Surfaces from a Single RGB Imagecited by 26

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 21; 2023: 36; 2024: 29; 2025: 20; 2026: 6.

125 papers.

Sort:

By area:

  1. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 389
  2. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
  3. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 357
  4. 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimationcited by 323
  5. 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 202
  6. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 170
  7. 2021SPViT: Enabling Faster Vision Transformers via Soft Token Pruningcited by 165
  8. 2021MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimationcited by 151
  9. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  10. 2022MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformercited by 136
  11. 2024Rethinking Inductive Biases for Surface Normal Estimationcited by 122
  12. 2022Attention Attention Everywhere: Monocular Depth Prediction with Skip Attentioncited by 92
38 more
  1. 2023DiffusionDepth: Diffusion Denoising Approach for Monocular Depth Estimationcited by 81
  2. 2024Align3R: Aligned Monocular Depth Estimation for Dynamic Videoscited by 79
  3. 2025Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysiscited by 78
  4. 2023iDisc: Internal Discretization for Monocular Depth Estimationcited by 71
  5. 2025UniK3D: Universal Camera Monocular 3D Estimationcited by 60
  6. 2023VA-DepthNet: A Variational Approach to Single Image Depth Predictioncited by 54
  7. 2021Shape from Polarization for Complex Scenes in the Wildcited by 53
  8. 2023LDM3D: Latent Diffusion Model for 3Dcited by 53
  9. 2023IEBins: Iterative Elastic Bins for Monocular Depth Estimationcited by 53
  10. 2023NDDepth: Normal-Distance Assisted Monocular Depth Estimationcited by 45
  11. 2022Interaction Transformer for Human Reaction Generationcited by 42
  12. 2023EGformer: Equirectangular Geometry-biased Transformer for 360 Depth Estimationcited by 42
  13. 2023PatchFusion: An End-to-End Tile-Based Framework for High-Resolution Monocular Metric Depth Estimationcited by 39
  14. 2024StereoCrafter: Diffusion-based Generation of Long and High-fidelity Stereoscopic 3D from Monocular Videoscited by 36
  15. 2025GeometryCrafter: Consistent Geometry Estimation for Open-world Videos with Diffusion Priorscited by 36
  16. 2023Improving Depth Gradient Continuity in Transformers: A Comparative Study on Monocular Depth Estimation with CNNcited by 35
  17. 2024WorDepth: Variational Language Prior for Monocular Depth Estimationcited by 34
  18. 2022Masked Jigsaw Puzzle: A Versatile Position Embedding for Vision Transformerscited by 33
  19. 2022MeshMAE: Masked Autoencoders for 3D Mesh Data Analysiscited by 31
  20. 2023FreeReg: Image-to-Point Cloud Registration Leveraging Pretrained Diffusion Models and Monocular Depth Estimatorscited by 29
  21. 2024Video Depth without Video Modelscited by 28
  22. 2024Flowing from Words to Pixels: A Noise-Free Framework for Cross-Modality Evolutioncited by 26
  23. 2022Deep Digging into the Generalization of Self-Supervised Monocular Depth Estimationcited by 24
  24. 2022Peeling the Onion: Hierarchical Reduction of Data Redundancy for Efficient Vision Transformer Trainingcited by 24
  25. 2023DiffI2I: Efficient Diffusion Model for Image-to-Image Translationcited by 24
  26. 2023Adaptive Spot-Guided Transformer for Consistent Local Feature Matchingcited by 22
  27. 2022Recent Advances in Vision Transformer: A Survey and Outlook of Recent Workcited by 21
  28. 2022GraphMLP: A Graph MLP-Like Architecture for 3D Human Pose Estimationcited by 21
  29. 2022Transformers in Self-Supervised Monocular Depth Estimation with Unknown Camera Intrinsicscited by 19
  30. 2022From 2D to 3D: Re-thinking Benchmarking of Monocular Depth Predictioncited by 19
  31. 2023Graph Transformer GANs for Graph-Constrained House Generationcited by 19
  32. 2023MonoATT: Online Monocular 3D Object Detection with Adaptive Token Transformercited by 18
  33. 2023Distilling Efficient Vision Transformers from CNNs for Semantic Segmentationcited by 18
  34. 2022Self-distilled Feature Aggregation for Self-supervised Monocular Depth Estimationcited by 17
  35. 2023URCDC-Depth: Uncertainty Rectified Cross-Distillation with CutFlip for Monocular Depth Estimationcited by 16
  36. 2022Continual Attentive Fusion for Incremental Learning in Semantic Segmentationcited by 15
  37. 2024Deep Learning-based Depth Estimation Methods from Monocular Image and Videos: A Comprehensive Surveycited by 15
  38. 2023Single Image Depth Prediction Made Better: A Multivariate Gaussian Takecited by 14

50 most cited of 125 All 125 (JSON)