mlsanityData to 14 Oct 2026

Depth Estimation with Simplified Transformer

arXiv 2204.13791 ↗First posted 28 Apr 2022cs.CV cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Transformer and its variants have shown state-of-the-art results in many vision tasks recently, ranging from image classification to dense prediction. Despite of their success, limited work has been reported on improving the model efficiency for deployment in latency-critical applications, such as autonomous driving and robotic navigation. In this paper, we aim at improving upon the existing transformers in vision, and propose a method for self-supervised monocular Depth Estimation with Simplified Transformer (DEST), which is efficient and particularly suitable for deployment on GPU-based platforms. Through strategic design choices, our model leads to significant reduction in model size, complexity, as well as inference latency, while achieving superior accuracy as compared to state-of-the-art. We also show that our design generalize well to other dense prediction task without bells and whistles.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 25 of its 33 references that are in the map. First posted on arXiv 28 Apr 2022.

Builds on

25 of 33 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  8. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  9. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  10. 2021Vision Transformers for Dense Predictioncited by 1,803
  11. 2020Rethinking Attention with Performerscited by 1,601
  12. 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
13 more
  1. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  2. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  3. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  4. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
  5. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 439
  6. 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
  7. 2020Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformerscited by 190
  8. 2018Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraintscited by 179
  9. 2015Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fieldscited by 169
  10. 2017Learning Depth from Monocular Videos using Direct Methodscited by 154
  11. 2021Efficient Self-supervised Vision Transformers for Representation Learningcited by 149
  12. 2020Towards Better Generalization: Joint Depth-Pose Learning without PoseNetcited by 85
  13. 2021Learning Efficient Vision Transformers via Fine-Grained Manifold Distillationcited by 60

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 3; 2024: 4.

8 papers.

  1. 2022Image Masking for Robust Self-Supervised Monocular Depth Estimationcited by 5
  2. 2023DeLiRa: Self-Supervised Depth, Light, and Radiance Fieldscited by 5
  3. 2024Towards Scale-Aware Full Surround Monodepth with Transformerscited by 5
  4. 2024A Novel Spike Transformer Network for Depth Estimation from Event Cameras via Cross-modality Knowledge Distillationcited by 2
  5. 2023FCN+: Global Receptive Convolution Makes FCN Great Againcited by 0
  6. 2023Swin-Free: Achieving Better Cross-Window Attention and Efficiency with Size-varying Windowcited by 0
  7. 2024Depth Estimation Algorithm Based on Transformer-Encoder and Feature Fusioncited by 0
  8. 2024ReduceFormer: Attention with Tensor Reduction by Summationcited by 0