mlsanityData to 14 Oct 2026

Visual Prompt Tuning for Generative Transfer Learning

arXiv 2210.00990 ↗First posted 3 Oct 2022cs.CV cs.AIcited by 75

Not on the map. Why?

Abstract (arXiv)

Transferring knowledge from an image synthesis model trained on a large dataset is a promising direction for learning generative image models from various domains efficiently. While previous works have studied GAN models, we present a recipe for learning vision transformers by generative knowledge transfer. We base our framework on state-of-the-art generative vision transformers that represent an image as a sequence of visual tokens to the autoregressive or non-autoregressive transformers. To adapt to a new domain, we employ prompt tuning, which prepends learnable tokens called prompt to the image token sequence, and introduce a new prompt design for our task. We study on a variety of visual domains, including visual task adaptation benchmark~citezhai2019large, with varying amount of training images, and show effectiveness of knowledge transfer and a significantly better image generation quality over existing works.

Key facts. Cited by 75 papers in the map (data through 14 Oct 2026). Builds on 73 of its 88 references that are in the map. First posted on arXiv 3 Oct 2022.

Builds on

73 of 88 references.

References · 73

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2020Language Models are Few-Shot Learnerscited by 32,419
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  7. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  8. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  9. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  10. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  11. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
38 more
  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  2. 2022Generative Adversarial Networkscited by 5,627
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  4. 2017Neural Discrete Representation Learningcited by 4,712
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  7. 2013Auto-Encoding Variational Bayescited by 4,053
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  10. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  11. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  12. 2015Fast R-CNNcited by 2,933
  13. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  14. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  15. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  16. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  17. 2022Visual Prompt Tuningcited by 1,681
  18. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  19. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
  20. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  21. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  22. 2019A Comprehensive Survey on Transfer Learningcited by 1,160
  23. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  24. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,013
  25. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  26. 2016Conditional Image Generation with PixelCNN Decoderscited by 862
  27. 2016Pixel Recurrent Neural Networkscited by 816
  28. 2018Image Transformercited by 772
  29. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  30. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 694
  31. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 689
  32. 2019A Closer Look at Few-shot Classificationcited by 684
  33. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  34. 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
  35. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 637
  36. 2017Learning multiple visual domains with residual adapterscited by 557
  37. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 492
  38. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 425

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 22; 2024: 31; 2025: 14; 2026: 5.

75 papers.

Sort:

By area:

  1. 2022MAGVIT: Masked Generative Video Transformercited by 346
  2. 2023StyleDrop: Text-to-Image Generation in Any Stylecited by 199
  3. 2023PromptIR: Prompting for All-in-One Blind Image Restorationcited by 114
  4. 2022Understanding and Improving Visual Prompting: A Label-Mapping Perspectivecited by 87
  5. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 83
  6. 2023CLIP goes 3D: Leveraging Prompt Tuning for Language Grounded 3D Recognitioncited by 77
  7. 2023Review of Large Vision Models and Visual Prompt Engineeringcited by 77
  8. 2024Affordance-based Robot Manipulation with Flow Matchingcited by 54
  9. 2024IRSAM: Advancing Segment Anything Model for Infrared Small Target Detectioncited by 46
  10. 2023From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videoscited by 39
  11. 2023Visual Tuningcited by 34
  12. 2024Mind the Interference: Retaining Pre-trained Knowledge in Parameter Efficient Continual Learning of Vision-Language Modelscited by 34
38 more
  1. 2024Facing the Elephant in the Room: Visual Prompt Tuning or Full Finetuning?cited by 31
  2. 2024Visual Fourier Prompt Tuningcited by 30
  3. 2023LoGoPrompt: Synthetic Text Images Can Be Good Visual Prompts for Vision-Language Modelscited by 29
  4. 2023AutoVP: An Automated Visual Prompting Framework and Benchmarkcited by 29
  5. 2023Tuning Multi-mode Token-level Prompt Alignment across Modalitiescited by 26
  6. 2023LeftRefill: Filling Right Canvas based on Left Reference through Generalized Text-to-Image Diffusion Modelcited by 25
  7. 2023Efficient Multimodal Semantic Segmentation via Dual-Prompt Learningcited by 25
  8. 2023Parameter-efficient is not sufficient: Exploring Parameter, Memory, and Time Efficient Adapter Tuning for Dense Predictionscited by 24
  9. 2024CPR: Retrieval Augmented Generation for Copyright Protectioncited by 24
  10. 2024Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey and Benchmarkcited by 23
  11. 2025Prompt-based Adaptation in Large-scale Vision Models: A Surveycited by 21
  12. 2024PromptCIR: Blind Compressed Image Restoration with Prompt Learningcited by 20
  13. 2023A Survey on Generative Modeling with Limited Data, Few Shots, and Zero Shotcited by 19
  14. 2024FlashST: A Simple and Universal Prompt-Tuning Framework for Traffic Predictioncited by 18
  15. 2023Training Data Protection with Compositional Diffusion Modelscited by 17
  16. 2023ZegOT: Zero-shot Segmentation Through Optimal Transport of Text Promptscited by 16
  17. 2023SPELL: Semantic Prompt Evolution based on a LLMcited by 15
  18. 2025Handling Out-of-Distribution Data: A Surveycited by 14
  19. 2023MedPrompt: Cross-Modal Prompting for Multi-Task Medical Image Translationcited by 13
  20. 2024Large Convolutional Model Tuning via Filter Subspacecited by 11
  21. 2023Learning Disentangled Prompts for Compositional Image Synthesiscited by 8
  22. 2024HCVP: Leveraging Hierarchical Contrastive Visual Prompt for Domain Generalizationcited by 8
  23. 2024Data-free Multi-label Image Recognition via LLM-powered Prompt Tuningcited by 8
  24. 2024PathoTune: Adapting Visual Foundation Model to Pathological Specialistscited by 8
  25. 2024UCIP: A Universal Framework for Compressed Image Super-Resolution using Dynamic Promptcited by 8
  26. 2025Foundation Models and Transformers for Anomaly Detection: A Surveycited by 8
  27. 2024Learning Visual Conditioning Tokens to Correct Domain Shift for Fully Test-time Adaptationcited by 7
  28. 2024FairQueue: Rethinking Prompt Learning for Fair Text-to-Image Generationcited by 7
  29. 2023MoP-CLIP: A Mixture of Prompt-Tuned CLIP Models for Domain Incremental Learningcited by 6
  30. 2025Revisit Visual Prompt Tuning: The Expressiveness of Prompt Expertscited by 5
  31. 2025CENet: Context Enhancement Network for Medical Image Segmentationcited by 5
  32. 2023Rethinking Visual Prompt Learning as Masked Visual Token Modelingcited by 4
  33. 2024Are You Copying My Prompt? Protecting the Copyright of Vision Prompt for VPaaS via Watermarkcited by 4
  34. 2024Dual-Path Adversarial Lifting for Domain Shift Correction in Online Test-time Adaptationcited by 4
  35. 2025Faster Parameter-Efficient Tuning with Token Redundancy Reductioncited by 4
  36. 2025Random Registers for Cross-Domain Few-Shot Learningcited by 4
  37. 2022Transferability Estimation Based On Principal Gradient Expectationcited by 3
  38. 2024SamLP: A Customized Segment Anything Model for License Plate Detectioncited by 3

50 most cited of 75 All 75 (JSON)