mlsanityData to 7 Oct 2026

MaPLe: Multi-modal Prompt Learning

arXiv 2210.03117 ↗First posted 6 Oct 2022cs.CVcited by 685

Not on the map. Why?

Cited by 326 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Pre-trained vision-language (V-L) models such as CLIP have shown excellent generalization ability to downstream tasks. However, they are sensitive to the choice of input text prompts and require careful selection of prompt templates to perform well. Inspired by the Natural Language Processing (NLP) literature, recent CLIP adaptation approaches learn prompts as the textual inputs to fine-tune CLIP for downstream tasks. We note that using prompting to adapt representations in a single branch of CLIP (language or vision) is sub-optimal since it does not allow the flexibility to dynamically adjust both representation spaces on a downstream task. In this work, we propose Multi-modal Prompt Learning (MaPLe) for both vision and language branches to improve alignment between the vision and language representations. Our design promotes strong coupling between the vision-language prompts to ensure mutual synergy and discourages learning independent uni-modal solutions. Further, we learn separate prompts across different early stages to progressively model the stage-wise feature relationships to allow rich context learning. We evaluate the effectiveness of our approach on three representative tasks of generalization to novel classes, new target datasets and unseen domain shifts. Compared with the state-of-the-art method Co-CoOp, MaPLe exhibits favorable performance and achieves an absolute gain of 3.45% on novel classes and 2.72% on overall harmonic-mean, averaged over 11 diverse image recognition datasets. Our code and pre-trained models are available at https://github.com/muzairkhattak/multimodal-prompt-learning.

Key facts. Cited by 685 papers in the map (data through 7 Oct 2026). Builds on 41 of its 53 references that are in the map. First posted on arXiv 6 Oct 2022.

Builds on

41 of 53 references.

References · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  6. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  7. 2013Describing Textures in the Wildcited by 2,110
  8. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  9. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  10. 2022Visual Prompt Tuningcited by 1,674
  11. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  12. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
29 more
  1. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  2. 2019Natural Adversarial Examplescited by 1,401
  3. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
  4. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  5. 2021Learning to Prompt for Continual Learningcited by 938
  6. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  7. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  8. 2022Language-driven Semantic Segmentationcited by 711
  9. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  10. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
  11. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  12. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
  13. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  14. 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 530
  15. 2021Image Segmentation Using Text and Image Promptscited by 509
  16. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 429
  17. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Modelscited by 379
  18. 2022Prompt-aligned Gradient for Prompt Tuningcited by 339
  19. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 309
  20. 2022Prompt Distribution Learningcited by 235
  21. 2022Open-Vocabulary DETR with Conditional Matchingcited by 220
  22. 2021Decoupling Zero-Shot Semantic Segmentationcited by 198
  23. 2021CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Modelscited by 182
  24. 2022Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detectioncited by 145
  25. 2022PromptDet: Towards Open-vocabulary Detection using Uncurated Imagescited by 133
  26. 2022Neural Prompt Searchcited by 133
  27. 2022Unsupervised Prompt Learning for Vision-Language Modelscited by 125
  28. 2021A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Modelscited by 101
  29. 2021Class-agnostic Object Detection with Multi-modal Transformercited by 85

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 99; 2024: 226; 2025: 237; 2026: 118.

685 papers.

Sort:

By area:

  1. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  2. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  3. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  4. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
  5. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 285
  6. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  7. 2023AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detectioncited by 270
  8. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 183
  9. 2022Fine-tuned CLIP Models are Efficient Video Learnerscited by 178
  10. 2024AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detectioncited by 126
  11. 2023DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Modelscited by 124
  12. 2023Multimodal Prompting with Missing Modalities for Visual Recognitioncited by 121
38 more
  1. 2023PromptIR: Prompting for All-in-One Blind Image Restorationcited by 113
  2. 2024Low-Rank Few-Shot Adaptation of Vision-Language Modelscited by 103
  3. 2023Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalizationcited by 100
  4. 2024PromptAD: Learning Prompts with only Normal Samples for Few-Shot Anomaly Detectioncited by 97
  5. 2022Understanding and Improving Visual Prompting: A Label-Mapping Perspectivecited by 86
  6. 2023CLIP for All Things Zero-Shot Sketch-Based Image Retrieval, Fine-Grained or Notcited by 79
  7. 2023Review of Large Vision Models and Visual Prompt Engineeringcited by 77
  8. 2023Consistency-guided Prompt Learning for Vision-Language Modelscited by 73
  9. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  10. 2024PromptKD: Unsupervised Prompt Distillation for Vision-Language Modelscited by 72
  11. 2023HIPTrack: Visual Tracking with Historical Promptscited by 71
  12. 2023BlackVIP: Black-Box Visual Prompting for Robust Transfer Learningcited by 70
  13. 2023GraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graphcited by 69
  14. 2024A Survey on Visual Anomaly Detection: Challenge, Approach, and Prospectcited by 69
  15. 2023A Closer Look at the Few-Shot Adaptation of Large Vision-Language Modelscited by 67
  16. 2024Sparse autoencoders reveal selective remapping of visual concepts during adaptationcited by 66
  17. 2023Single-Model and Any-Modality for Video Object Trackingcited by 64
  18. 2023BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIPcited by 63
  19. 2023AD-CLIP: Adapting Domains in Prompt Space Using CLIPcited by 62
  20. 2024Explore the Potential of CLIP for Training-Free Open Vocabulary Semantic Segmentationcited by 61
  21. 2023Sentence-level Prompts Benefit Composed Image Retrievalcited by 59
  22. 2024A Multimodal Knowledge-enhanced Whole-slide Pathology Foundation Modelcited by 59
  23. 2023DePT: Decoupled Prompt Tuningcited by 58
  24. 2024A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trendscited by 58
  25. 2024Improved Zero-Shot Classification by Adapting VLMs with Text Descriptionscited by 52
  26. 2024Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Modelscited by 51
  27. 2024Global and Local Prompts Cooperation via Optimal Transport for Federated Learningcited by 45
  28. 2023CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformerscited by 44
  29. 2024Learning Transferable Negative Prompts for Out-of-Distribution Detectioncited by 43
  30. 2025Parameter-Efficient Fine-Tuning for Foundation Modelscited by 43
  31. 2023Adversarial Prompt Tuning for Vision-Language Modelscited by 42
  32. 2023CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Updatecited by 42
  33. 2024CARZero: Cross-Attention Alignment for Radiology Zero-Shot Classificationcited by 40
  34. 2023ArGue: Attribute-Guided Prompt Tuning for Vision-Language Modelscited by 39
  35. 2024Few-Shot Adversarial Prompt Learning on Vision-Language Modelscited by 39
  36. 2024TempMe: Video Temporal Token Merging for Efficient Text-Video Retrievalcited by 39
  37. 2024Recent Advances of Multimodal Continual Learning: A Comprehensive Surveycited by 39
  38. 2023SD4Match: Learning to Prompt Stable Diffusion Model for Semantic Matchingcited by 38

50 most cited of 685 All 685 (JSON)