mlsanityData to 7 Oct 2026

Vision Transformers for Dense Prediction

arXiv 2103.13413 ↗First posted 24 Mar 2021cs.CVcited by 1,789

Not on the map. Why?

Cited by 130 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce dense vision transformers, an architecture that leverages vision transformers in place of convolutional networks as a backbone for dense prediction tasks. We assemble tokens from various stages of the vision transformer into image-like representations at various resolutions and progressively combine them into full-resolution predictions using a convolutional decoder. The transformer backbone processes representations at a constant and relatively high resolution and has a global receptive field at every stage. These properties allow the dense vision transformer to provide finer-grained and more globally coherent predictions when compared to fully-convolutional networks. Our experiments show that this architecture yields substantial improvements on dense prediction tasks, especially when a large amount of training data is available. For monocular depth estimation, we observe an improvement of up to 28% in relative performance when compared to a state-of-the-art fully-convolutional network. When applied to semantic segmentation, dense vision transformers set a new state of the art on ADE20K with 49.02% mIoU. We further show that the architecture can be fine-tuned on smaller datasets such as NYUv2, KITTI, and Pascal Context where it also sets the new state of the art. Our models are available at https://github.com/intel-isl/DPT.

Key facts. Cited by 1,789 papers in the map (data through 7 Oct 2026). Builds on 49 of its 60 references that are in the map. First posted on arXiv 24 Mar 2021.

Builds on

49 of 60 references.

References · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  7. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  8. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  9. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  11. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  12. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
37 more
  1. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  2. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  3. 2016Pyramid Scene Parsing Networkcited by 2,472
  4. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  5. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  6. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
  7. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  8. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  9. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  10. 2019Objects as Pointscited by 1,172
  11. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
  12. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  13. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
  14. 2018Image Transformercited by 770
  15. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  16. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
  17. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  18. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  19. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  20. 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 573
  21. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  22. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  23. 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 487
  24. 2016RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentationcited by 463
  25. 2020ResNeSt: Split-Attention Networkscited by 445
  26. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 379
  27. 2015Learning Deconvolution Network for Semantic Segmentationcited by 365
  28. 2019Attention Augmented Convolutional Networkscited by 363
  29. 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
  30. 2020Exploring Self-attention for Image Recognitioncited by 313
  31. 2019Enforcing geometric constraints of virtual normal for depth predictioncited by 233
  32. 2013OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networkscited by 224
  33. 2018The ApolloScape Open Dataset for Autonomous Driving and its Applicationcited by 188
  34. 2019Learning the Depths of Moving People by Watching Frozen Peoplecited by 120
  35. 2019Web Stereo Video Supervision for Depth Prediction from Dynamic Scenescited by 73
  36. 2019Adaptive Context Network for Scene Parsingcited by 51
  37. 2019IRS: A Large Naturalistic Indoor Robotics Stereo Dataset to Train Deep Models for Disparity and Surface Normal Estimationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 60; 2022: 205; 2023: 315; 2024: 331; 2025: 450; 2026: 428.

1,789 papers.

Sort:

By area:

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  3. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  4. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  5. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  6. 2024Depth Anything V2cited by 1,385
  7. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  8. 2025Qwen-Image Technical Reportcited by 973
  9. 2024Grounding Image Matching in 3D with MASt3Rcited by 960
  10. 2021Uformer: A General U-Shaped Transformer for Image Restorationcited by 864
  11. 2022Language-driven Semantic Segmentationcited by 711
  12. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
38 more
  1. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  2. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  3. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  4. 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 636
  5. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  6. 2022Vision Transformer Adapter for Dense Predictionscited by 564
  7. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  8. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  9. 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 467
  10. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 431
  11. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  12. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 395
  13. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 389
  14. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
  15. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  16. 2023Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fieldscited by 361
  17. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 357
  18. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 356
  19. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  20. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  21. 2021XCiT: Cross-Covariance Image Transformerscited by 335
  22. 2023GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splattingcited by 331
  23. 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimationcited by 323
  24. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 315
  25. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 303
  26. 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 295
  27. 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 290
  28. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 287
  29. 2023Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistlescited by 287
  30. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Imagecited by 283
  31. 2023FSGS: Real-Time Few-shot View Synthesis using Gaussian Splattingcited by 278
  32. 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 275
  33. 20243D Reconstruction with Spatial Memorycited by 269
  34. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 266
  35. 2024DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalizationcited by 256
  36. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
  37. 2022SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentationcited by 252
  38. 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 250

50 most cited of 1,789 All 1,789 (JSON)