mlsanityData to 14 Oct 2026

Vision Transformer Adapter for Dense Predictions

arXiv 2205.08534 ↗First posted 17 May 2022cs.CVcited by 565

Not on the map. Why?

Cited by 24 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This work investigates a simple yet powerful dense prediction task adapter for Vision Transformer (ViT). Unlike recently advanced variants that incorporate vision-specific inductive biases into their architectures, the plain ViT suffers inferior performance on dense predictions due to weak prior assumptions. To address this issue, we propose the ViT-Adapter, which allows plain ViT to achieve comparable performance to vision-specific transformers. Specifically, the backbone in our framework is a plain ViT that can learn powerful representations from large-scale multi-modal data. When transferring to downstream tasks, a pre-training-free adapter is used to introduce the image-related inductive biases into the model, making it suitable for these tasks. We verify ViT-Adapter on multiple dense prediction tasks, including object detection, instance segmentation, and semantic segmentation. Notably, without using extra detection data, our ViT-Adapter-L yields state-of-the-art 60.9 box AP and 53.0 mask AP on COCO test-dev. We hope that the ViT-Adapter could serve as an alternative for vision-specific transformers and facilitate future research. The code and models will be released at https://github.com/czczup/ViT-Adapter.

Key facts. Cited by 565 papers in the map (data through 14 Oct 2026). Builds on 83 of its 109 references that are in the map. First posted on arXiv 17 May 2022.

Builds on

83 of 109 references.

References · 83

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2017Mask R-CNNcited by 7,277
  9. 2016Layer Normalizationcited by 5,669
  10. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  11. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  12. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
38 more
  1. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  2. 2022A ConvNet for the 2020scited by 3,579
  3. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  4. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  5. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  6. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  7. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
  8. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  9. 2021Vision Transformers for Dense Predictioncited by 1,803
  10. 2022Visual Prompt Tuningcited by 1,681
  11. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
  12. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  13. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  14. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  15. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  16. 2021Video Swin Transformercited by 1,091
  17. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,026
  18. 2016Deep Networks with Stochastic Depthcited by 979
  19. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  20. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  21. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  22. 2021Going deeper with Image Transformerscited by 764
  23. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 751
  24. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
  25. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
  26. 2019Panoptic Feature Pyramid Networkscited by 645
  27. 2021Transformer in Transformercited by 633
  28. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 596
  29. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 569
  30. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  31. 2017Learning multiple visual domains with residual adapterscited by 557
  32. 2019Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selectioncited by 540
  33. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 527
  34. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 519
  35. 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 495
  36. 2019Hybrid Task Cascade for Instance Segmentationcited by 481
  37. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 431
  38. 2019Cascade R-CNN: High Quality Object Detection and Instance Segmentationcited by 429

50 most cited of 83 All 83 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 29; 2023: 141; 2024: 160; 2025: 142; 2026: 93.

565 papers.

Sort:

By area:

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  2. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  3. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  4. 2024Depth Anything V2cited by 1,422
  5. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  6. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  7. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 982
  8. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  9. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
  10. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 656
  11. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 565
  12. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 511
38 more
  1. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 501
  2. 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 482
  3. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 404
  4. 2023EVA-02: A Visual Representation for Neon Genesiscited by 381
  5. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 328
  6. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 317
  7. 2022Generalized Decoding for Pixel, Image, and Languagecited by 294
  8. 2023Personalize Segment Anything Model with One Shotcited by 250
  9. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  10. 2023Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIPcited by 201
  11. 2025FoundationStereo: Zero-Shot Stereo Matchingcited by 173
  12. 2024TokenPacker: Efficient Visual Projector for Multimodal LLMcited by 172
  13. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 165
  14. 2024ControlNeXt: Powerful and Efficient Control for Image and Video Generationcited by 159
  15. 2023BiFormer: Vision Transformer with Bi-Level Routing Attentioncited by 151
  16. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  17. 2024MambaOut: Do We Really Need Mamba for Vision?cited by 143
  18. 2024PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognitioncited by 141
  19. 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
  20. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  21. 2023SAM-Med2Dcited by 124
  22. 2022GRiT: A Generative Region-to-text Transformer for Object Understandingcited by 121
  23. 2023ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalitiescited by 118
  24. 2022UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormercited by 116
  25. 2023The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open Worldcited by 116
  26. 2022Open Vocabulary Semantic Segmentation with Patch Aligned Contrastive Learningcited by 115
  27. 2023EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modelingcited by 105
  28. 2022HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutionscited by 103
  29. 2023Towards image compression with perfect realism at ultra-low bitratescited by 103
  30. 2024Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architecturescited by 98
  31. 2023SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentationcited by 97
  32. 2023InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Languagecited by 96
  33. 2022Contrastive Learning Rivals Masked Image Modeling in Fine-tuning via Feature Distillationcited by 93
  34. 2024MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizercited by 80
  35. 2024Open-Vocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactivelycited by 75
  36. 2024OMG-Seg: Is One Model Good Enough For All Segmentation?cited by 75
  37. 2023Consistency-guided Prompt Learning for Vision-Language Modelscited by 73
  38. 2022Vision Transformers are Parameter-Efficient Audio-Visual Learnerscited by 72

50 most cited of 565 All 565 (JSON)