mlsanityData to 14 Oct 2026

E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuning

arXiv 2307.13770 ↗First posted 25 Jul 2023cs.CV cs.AIcited by 70

Not on the map. Why?

Abstract (arXiv)

As the size of transformer-based models continues to grow, fine-tuning these large-scale pretrained vision models for new tasks has become increasingly parameter-intensive. Parameter-efficient learning has been developed to reduce the number of tunable parameters during fine-tuning. Although these methods show promising results, there is still a significant performance gap compared to full fine-tuning. To address this challenge, we propose an Effective and Efficient Visual Prompt Tuning (E^2VPT) approach for large-scale transformer-based model adaptation. Specifically, we introduce a set of learnable key-value prompts and visual prompts into self-attention and input layers, respectively, to improve the effectiveness of model fine-tuning. Moreover, we design a prompt pruning procedure to systematically prune low importance prompts while preserving model performance, which largely enhances the model's efficiency. Empirical results demonstrate that our approach outperforms several state-of-the-art baselines on two benchmarks, with considerably low parameter usage (e.g., 0.32% of model parameters on VTAB-1k). Our code is available at https://github.com/ChengHan111/E2VPT.

Key facts. Cited by 70 papers in the map (data through 14 Oct 2026). Builds on 81 of its 105 references that are in the map. First posted on arXiv 25 Jul 2023.

Builds on

81 of 105 references.

References · 81

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  8. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  9. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  10. 2020End-to-End Object Detection with Transformerscited by 7,414
  11. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  12. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
38 more
  1. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  2. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  3. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  4. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  5. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  6. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  7. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  8. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  9. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  10. 2022Visual Prompt Tuningcited by 1,681
  11. 2021ViViT: A Video Vision Transformercited by 1,620
  12. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  13. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  14. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  15. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  16. 2014How transferable are features in deep neural networks?cited by 1,225
  17. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  18. 2016Colorful Image Colorizationcited by 1,114
  19. 2021Transformers in Vision: A Surveycited by 1,013
  20. 2021Scaling Vision Transformerscited by 999
  21. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  22. 2019Are Sixteen Heads Really Better than One?cited by 816
  23. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  24. 2021Scaling Vision with Sparse Mixture of Expertscited by 747
  25. 2019Large Batch Optimization for Deep Learning: Training BERT in 76 minutescited by 674
  26. 2020A Survey on Visual Transformercited by 654
  27. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 605
  28. 2017Learning multiple visual domains with residual adapterscited by 557
  29. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 519
  30. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  31. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 425
  32. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 418
  33. 2018Hyperbolic Neural Networkscited by 402
  34. 2021A Survey of Transformerscited by 401
  35. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  36. 2020Parameter-Efficient Transfer Learning with Diff Pruningcited by 352
  37. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 310
  38. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 24; 2025: 32; 2026: 12.

70 papers.

Sort:

By area:

  1. 2024PromptKD: Unsupervised Prompt Distillation for Vision-Language Modelscited by 72
  2. 2024Revisiting the Power of Prompt for Visual Tuningcited by 32
  3. 2024Facing the Elephant in the Room: Visual Prompt Tuning or Full Finetuning?cited by 31
  4. 2024Visual Fourier Prompt Tuningcited by 30
  5. 2025Visual Instance-aware Prompt Tuningcited by 29
  6. 2025ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classificationcited by 28
  7. 2023Efficient Multimodal Semantic Segmentation via Dual-Prompt Learningcited by 25
  8. 2024Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey and Benchmarkcited by 23
  9. 2024Adapting to Distribution Shift by Visual Domain Prompt Generationcited by 23
  10. 2025Prompt-based Adaptation in Large-scale Vision Models: A Surveycited by 21
  11. 2024Sample-specific Masks for Visual Reprogramming-based Promptingcited by 18
  12. 2023STA-VPT: SpatioTemporally Aligned Visual Prompt Tuningcited by 17
38 more
  1. 2025DA-VPT: Semantic-Guided Visual Prompt Tuning for Vision Transformerscited by 17
  2. 2024ExPLoRA: Parameter-Efficient Extended Pre-Training to Adapt Vision Transformers under Domain Shiftscited by 16
  3. 2024M2PT: Multimodal Prompt Tuning for Zero-shot Instruction Learningcited by 16
  4. 2024Improving Visual Prompt Tuning by Gaussian Neighborhood Minimization for Long-Tailed Visual Recognitioncited by 15
  5. 2025Learning to Adapt Frozen CLIP for Few-Shot Test-Time Domain Adaptationcited by 14
  6. 2024Enhancing 3D Object Detection with 2D Detection-Guided Query Anchorscited by 12
  7. 2024AMD: Automatic Multi-step Distillation of Large-scale Vision Modelscited by 12
  8. 2024Unsupervised Domain Adaption Harnessing Vision-Language Pre-trainingcited by 12
  9. 2024Bayesian-guided Label Mapping for Visual Reprogrammingcited by 12
  10. 2024Parameter-Efficient Fine-Tuning of Large Language Models using Semantic Knowledge Tuningcited by 10
  11. 2025Visual Variational Autoencoder Prompt Tuningcited by 10
  12. 2025Re-Imagining Multimodal Instruction Tuning: A Representation Viewcited by 9
  13. 2025PRO-VPT: Distribution-Adaptive Visual Prompt Tuning via Prompt Relocationcited by 9
  14. 2025Attribute-based Visual Reprogramming for Vision-Language Modelscited by 8
  15. 2025Revisiting Pool-based Prompt Learning for Few-shot Class-incremental Learningcited by 8
  16. 2025All You Need is One: Capsule Prompt Tuning with a Single Vectorcited by 8
  17. 2025Efficient Multi-modal Long Context Learning for Training-free Adaptationcited by 7
  18. 2024Token Compensator: Altering Inference Cost of Vision Transformer without Re-Tuningcited by 6
  19. 2025CIBR: Cross-modal Information Bottleneck Regularization for Robust CLIP Generalizationcited by 6
  20. 2024Image Translation as Diffusion Visual Programmerscited by 5
  21. 2025Revisit Visual Prompt Tuning: The Expressiveness of Prompt Expertscited by 5
  22. 2025MEPT: Mixture of Expert Prompt Tuning as a Manifold Mappercited by 5
  23. 2025UniUltra: Interactive Parameter-Efficient SAM2 for Universal Ultrasound Segmentationcited by 5
  24. 2024iVPT: Improving Task-relevant Information Sharing in Visual Prompt Tuning by Cross-layer Dynamic Connectioncited by 4
  25. 2024Prototypical Transformer as Unified Motion Learnerscited by 4
  26. 2025Correlative and Discriminative Label Grouping for Multi-Label Visual Prompt Tuningcited by 4
  27. 2025PrefixMemory-Tuning: Modernizing Prefix-Tuning by Decoupling the Prefix from Attentioncited by 4
  28. 2025Attention to the Burstiness in Visual Prompt Tuning!cited by 4
  29. 2024MLP: Motion Label Prior for Temporal Sentence Localization in Untrimmed 3D Human Motionscited by 3
  30. 2025Token Coordinated Prompt Attention is Needed for Visual Promptingcited by 3
  31. 2025Vision Graph Prompting via Semantic Low-Rank Decompositioncited by 3
  32. 2025Self-Supervised Visual Prompting for Cross-Domain Road Damage Detectioncited by 3
  33. 2024CVPT: Cross Visual Prompt Tuningcited by 2
  34. 2024Multiple-Exit Tuning: Towards Inference-Efficient Adaptation for Vision Transformercited by 2
  35. 2024AdaptGCD: Multi-Expert Adapter Tuning for Generalized Category Discoverycited by 2
  36. 2025Exploring Interpretability for Visual Prompt Tuning with Cross-layer Conceptscited by 2
  37. 2025State Space Prompting via Gathering and Spreading Spatio-Temporal Information for Video Understandingcited by 2
  38. 2026Visual Prompt-Agnostic Evolutioncited by 2

50 most cited of 70 All 70 (JSON)