mlsanityData to 7 Oct 2026

Neural Prompt Search

arXiv 2206.04673 ↗First posted 9 Jun 2022cs.CV cs.AI cs.LGcited by 133

Not on the map. Why?

Abstract (arXiv)

The size of vision models has grown exponentially over the last few years, especially after the emergence of Vision Transformer. This has motivated the development of parameter-efficient tuning methods, such as learning adapter layers or visual prompt tokens, which allow a tiny portion of model parameters to be trained whereas the vast majority obtained from pre-training are frozen. However, designing a proper tuning method is non-trivial: one might need to try out a lengthy list of design choices, not to mention that each downstream dataset often requires custom designs. In this paper, we view the existing parameter-efficient tuning methods as "prompt modules" and propose Neural prOmpt seArcH (NOAH), a novel approach that learns, for large vision models, the optimal design of prompt modules through a neural architecture search algorithm, specifically for each downstream dataset. By conducting extensive experiments on over 20 vision datasets, we demonstrate that NOAH (i) is superior to individual prompt modules, (ii) has a good few-shot learning ability, and (iii) is domain-generalizable. The code and models are available at https://github.com/Davidzhangyuanhan/NOAH.

Key facts. Cited by 133 papers in the map (data through 7 Oct 2026). Builds on 48 of its 70 references that are in the map. First posted on arXiv 9 Jun 2022.

Builds on

48 of 70 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  9. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  11. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  12. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
36 more
  1. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  2. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  3. 2013Describing Textures in the Wildcited by 2,110
  4. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  6. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  7. 2022Visual Prompt Tuningcited by 1,674
  8. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  9. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  10. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  11. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  12. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  13. 2019Natural Adversarial Examplescited by 1,401
  14. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  15. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  16. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
  17. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  18. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  19. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  20. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  21. 2022A Generalist Agentcited by 802
  22. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 791
  23. 2021Domain Generalization: A Surveycited by 760
  24. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  25. 2018Rotation Equivariant CNNs for Digital Pathologycited by 420
  26. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 419
  27. 2017Large-Scale Evolution of Image Classifierscited by 348
  28. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  29. 2022Prompt Distribution Learningcited by 235
  30. 2021AutoFormer: Searching Transformers for Visual Recognitioncited by 186
  31. 2021UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuningcited by 162
  32. 2021Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuningcited by 145
  33. 2022Revisiting Parameter-Efficient Tuning: Are We Really There Yet?cited by 76
  34. 2019Learning Generalisable Omni-Scale Representations for Person Re-Identificationcited by 73
  35. 2020AttentiveNAS: Improving Neural Architecture Search via Attentive Samplingcited by 65
  36. 2022Bamboo: Building Mega-Scale Vision Dataset Continually with Human-Machine Synergycited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 11; 2023: 45; 2024: 48; 2025: 21; 2026: 8.

133 papers.

Sort:

By area:

  1. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  2. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
  3. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  4. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 183
  5. 2023What Makes Good Examples for Visual In-Context Learning?cited by 150
  6. 2022FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformercited by 133
  7. 2022Convolutional Bypasses Are Better Vision Transformer Adapterscited by 120
  8. 2023Parameter-Efficient Orthogonal Finetuning via Butterfly Factorizationcited by 112
  9. 2023One-for-All: Generalized LoRA for Parameter-Efficient Fine-tuningcited by 84
  10. 2023Sensitivity-Aware Visual Parameter-Efficient Fine-Tuningcited by 74
  11. 2023Towards Efficient Visual Adaption via Structural Re-parameterizationcited by 73
  12. 2022Visual Query Tuning: Towards Effective Usage of Intermediate Representations for Parameter and Memory Efficient Transfer Learningcited by 57
38 more
  1. 2022Dual Modality Prompt Tuning for Vision-Language Pre-Trained Modelcited by 51
  2. 2022Texts as Images in Prompt Tuning for Multi-Label Image Recognitioncited by 51
  3. 2024One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Modelscited by 48
  4. 2023Revisiting the Parameter Efficiency of Adapters from the Perspective of Precision Redundancycited by 44
  5. 2024Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologiescited by 44
  6. 2025Parameter-Efficient Fine-Tuning for Foundation Modelscited by 43
  7. 2022Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Modelscited by 40
  8. 2024TempMe: Video Temporal Token Merging for Efficient Text-Video Retrievalcited by 39
  9. 2023FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Taskscited by 37
  10. 2024Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Surveycited by 36
  11. 2023Efficient Adaptation of Large Vision Transformer via Adapter Re-Composingcited by 35
  12. 2023Visual Tuningcited by 34
  13. 2022iQuery: Instruments as Queries for Audio-Visual Sound Separationcited by 29
  14. 2024AUFormer: Vision Transformers are Parameter-Efficient Facial Action Unit Detectorscited by 29
  15. 2024Time-, Memory- and Parameter-Efficient Visual Adaptationcited by 28
  16. 2024MoE-FFD: Mixture of Experts for Generalized and Parameter-Efficient Face Forgery Detectioncited by 27
  17. 2022FiT: Parameter Efficient Few-shot Transfer Learning for Personalized and Federated Image Classificationcited by 26
  18. 2023Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbonecited by 26
  19. 2022Towards a Unified View on Visual Parameter-Efficient Transfer Learningcited by 25
  20. 2024Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptationcited by 25
  21. 2023Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspectivecited by 23
  22. 2025AuroRA: Breaking Low-Rank Bottleneck of LoRA with Nonlinear Mappingcited by 22
  23. 2024Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey and Benchmarkcited by 21
  24. 2023SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channelscited by 20
  25. 2024Memory-Space Visual Prompting for Efficient Vision-Language Fine-Tuningcited by 20
  26. 2023Consolidator: Mergeable Adapter with Grouped Connections for Visual Adaptationcited by 18
  27. 2024Eureka: Evaluating and Understanding Large Foundation Modelscited by 18
  28. 2023Hydra: Multi-head Low-rank Adaptation for Parameter Efficient Fine-tuningcited by 17
  29. 2024PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptationcited by 17
  30. 2025DA-VPT: Semantic-Guided Visual Prompt Tuning for Vision Transformerscited by 17
  31. 2023TransHP: Image Classification with Hierarchical Promptingcited by 16
  32. 2024Adapters Strike Backcited by 16
  33. 2024Lessons and Insights from a Unifying Study of Parameter-Efficient Fine-Tuning (PEFT) in Visual Recognitioncited by 16
  34. 2025Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Modelcited by 15
  35. 2023ZeroI2V: Zero-Cost Adaptation of Pre-trained Transformers from Image to Videocited by 14
  36. 2024PACE: Marrying generalization in PArameter-efficient fine-tuning with Consistency rEgularizationcited by 14
  37. 2022Prompt Tuning with Soft Context Sharing for Vision-Language Modelscited by 13
  38. 2024Adapter-X: A Novel General Parameter-Efficient Fine-Tuning Framework for Visioncited by 13

50 most cited of 133 All 133 (JSON)