mlsanityData to 7 Oct 2026

Visual Prompt Tuning

arXiv 2203.12119 ↗First posted 23 Mar 2022cs.CVcited by 1,674

Not on the map. Why?

Cited by 292 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The current modus operandi in adapting pre-trained models involves updating all the backbone parameters, ie, full fine-tuning. This paper introduces Visual Prompt Tuning (VPT) as an efficient and effective alternative to full fine-tuning for large-scale Transformer models in vision. Taking inspiration from recent advances in efficiently tuning large language models, VPT introduces only a small amount (less than 1% of model parameters) of trainable parameters in the input space while keeping the model backbone frozen. Via extensive experiments on a wide variety of downstream recognition tasks, we show that VPT achieves significant performance gains compared to other parameter efficient tuning protocols. Most importantly, VPT even outperforms full fine-tuning in many cases across model capacities and training data scales, while reducing per-task storage cost.

Key facts. Cited by 1,674 papers in the map (data through 7 Oct 2026). Builds on 62 of its 97 references that are in the map. First posted on arXiv 23 Mar 2022.

Builds on

62 of 97 references.

References · 62

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  8. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  9. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  10. 2020End-to-End Object Detection with Transformerscited by 7,384
  11. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  12. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
38 more
  1. 2016Layer Normalizationcited by 5,579
  2. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  7. 2022A ConvNet for the 2020scited by 3,545
  8. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  9. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  10. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  11. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  12. 2013Describing Textures in the Wildcited by 2,110
  13. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  14. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  15. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  16. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  17. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  18. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  19. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  20. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  21. 2014How transferable are features in deep neural networks?cited by 1,216
  22. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
  23. 2016Colorful Image Colorizationcited by 1,113
  24. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  25. 2019How Can We Know What Language Models Know?cited by 953
  26. 2021Learning to Prompt for Continual Learningcited by 938
  27. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  28. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  29. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 791
  30. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  31. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  32. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  33. 2017Learning multiple visual domains with residual adapterscited by 553
  34. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  35. 2018Rotation Equivariant CNNs for Digital Pathologycited by 420
  36. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 419
  37. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  38. 2020Parameter-Efficient Transfer Learning with Diff Pruningcited by 351

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 76; 2023: 385; 2024: 479; 2025: 476; 2026: 258.

1,674 papers.

Sort:

By area:

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  3. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  4. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  5. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  6. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  7. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  8. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  9. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  10. 2022Vision Transformer Adapter for Dense Predictionscited by 564
  11. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 560
  12. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
38 more
  1. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  2. 2023Segment Anything in High Qualitycited by 373
  3. 2023Visual-Language Prompt Tuning with Knowledge-guided Context Optimizationcited by 307
  4. 2022Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuningcited by 292
  5. 2023Customized Segment Anything Model for Medical Image Segmentationcited by 277
  6. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  7. 2022Exploring Visual Prompts for Adapting Large-Scale Modelscited by 271
  8. 2023AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detectioncited by 270
  9. 2023Personalize Segment Anything Model with One Shotcited by 250
  10. 2024TextGrad: Automatic "Differentiation" via Textcited by 250
  11. 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
  12. 2022LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learningcited by 228
  13. 2023CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentationcited by 215
  14. 2023What does CLIP know about a red circle? Visual prompt engineering for VLMscited by 215
  15. 2023Class-Incremental Learning: A Surveycited by 209
  16. 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 200
  17. 2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapterscited by 199
  18. 2022ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learningcited by 192
  19. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 191
  20. 2024InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learningcited by 189
  21. 2022PointCLIP V2: Prompting CLIP and GPT for Powerful 3D Open-world Learningcited by 186
  22. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 183
  23. 2023RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Modelcited by 180
  24. 2023Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detectioncited by 180
  25. 2022ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentationcited by 178
  26. 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 162
  27. 2022CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-trainingcited by 159
  28. 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
  29. 2024Continual Learning with Pre-Trained Models: A Surveycited by 151
  30. 2023Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuningcited by 150
  31. 2022Unified Vision and Language Prompt Learningcited by 145
  32. 2023AIM: Adapting Image Models for Efficient Video Action Recognitioncited by 145
  33. 2022ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Modelscited by 141
  34. 2023Visual Prompt Multi-Modal Trackingcited by 141
  35. 2022Neural Prompt Searchcited by 133
  36. 2022FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformercited by 133
  37. 2023CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matchingcited by 133
  38. 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interactioncited by 133

50 most cited of 1,674 All 1,674 (JSON)