Dynamic Visual Prompt Tuning for Parameter Efficient Transfer Learning
Not on the map. Why?
Abstract (arXiv)
Parameter efficient transfer learning (PETL) is an emerging research spot that aims to adapt large-scale pre-trained models to downstream tasks. Recent advances have achieved great success in saving storage and computation costs. However, these methods do not take into account instance-specific visual clues for visual tasks. In this paper, we propose a Dynamic Visual Prompt Tuning framework (DVPT), which can generate a dynamic instance-wise token for each image. In this way, it can capture the unique visual feature of each image, which can be more suitable for downstream visual tasks. We designed a Meta-Net module that can generate learnable prompts based on each image, thereby capturing dynamic instance-wise visual features. Extensive experiments on a wide range of downstream recognition tasks show that DVPT achieves superior performance than other PETL methods. More importantly, DVPT even outperforms full fine-tuning on 17 out of 19 downstream tasks while maintaining high parameter efficiency. Our code will be released soon.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 32 of its 36 references that are in the map. First posted on arXiv 12 Sep 2023.
Builds on
32 of 36 references.
References · 32
- 2017Attention Is All You Needcited by 60,361
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2020End-to-End Object Detection with Transformerscited by 7,414
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
- 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
- 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
- 2021Learning to Prompt for Vision-Language Modelscited by 2,281
- 2021SwinIR: Image Restoration Using Swin Transformercited by 1,887
20 more
- 2022Visual Prompt Tuningcited by 1,681
- 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
- 2021ViViT: A Video Vision Transformercited by 1,620
- 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
- 2020Point Transformercited by 1,470
- 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
- 2020Pre-Trained Image Processing Transformercited by 893
- 2020PCT: Point cloud transformercited by 732
- 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 719
- 2018Do Better ImageNet Models Transfer Better?cited by 565
- 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 460
- 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 425
- 2022Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuningcited by 294
- 2021TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Upcited by 162
- 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
- 2022Neural Prompt Searchcited by 135
- 2021Generative Adversarial Transformerscited by 120
- 2022Head2Toe: Utilizing Intermediate Representations for Better Transfer Learningcited by 83
- 2022Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulationcited by 49
- 2021MVT: Multi-view Vision Transformer for 3D Object Recognitioncited by 34
Built on by
Citing papers per year: 2024: 1; 2025: 2.
3 papers.