mlsanityData to 7 Oct 2026

CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailment

arXiv 2203.07190 ↗First posted 14 Mar 2022cs.CV cs.CLcited by 113

Not on the map. Why?

Cited by 38 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

CLIP has shown a remarkable zero-shot capability on a wide range of vision tasks. Previously, CLIP is only regarded as a powerful visual encoder. However, after being pre-trained by language supervision from a large amount of image-caption pairs, CLIP itself should also have acquired some few-shot abilities for vision-language tasks. In this work, we empirically show that CLIP can be a strong vision-language few-shot learner by leveraging the power of language. We first evaluate CLIP's zero-shot performance on a typical visual question answering task and demonstrate a zero-shot cross-modality transfer capability of CLIP on the visual entailment task. Then we propose a parameter-efficient fine-tuning strategy to boost the few-shot performance on the vqa task. We achieve competitive zero/few-shot results on the visual question answering and visual entailment tasks without introducing any additional pre-training procedure.

Key facts. Cited by 113 papers in the map (data through 7 Oct 2026). Builds on 34 of its 54 references that are in the map. First posted on arXiv 14 Mar 2022.

Builds on

34 of 54 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  9. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  10. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  11. 2015VQA: Visual Question Answeringcited by 2,489
  12. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
22 more
  1. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  2. 2019Language Models as Knowledge Bases?cited by 1,922
  3. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  4. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  5. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  6. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  7. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  8. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  9. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  10. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  11. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
  12. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  13. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  14. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 542
  15. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 424
  16. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 376
  17. 2019Deep Modular Co-Attention Networks for Visual Question Answeringcited by 338
  18. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  19. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  20. 2019Visual Entailment: A Novel Task for Fine-Grained Image Understandingcited by 250
  21. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 246
  22. 2018Transforming Question Answering Datasets Into Natural Language Inference Datasetscited by 99

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 15; 2023: 34; 2024: 36; 2025: 19; 2026: 9.

113 papers.

Sort:

By area:

  1. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  2. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
  3. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
  4. 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensingcited by 357
  5. 2023Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrievalcited by 155
  6. 2023Weakly Supervised 3D Open-vocabulary Segmentationcited by 133
  7. 2022Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Modelscited by 95
  8. 2022Text-Only Training for Image Captioning using Noise-Injected CLIPcited by 91
  9. 2023Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Modelscited by 91
  10. 2023Open-Vocabulary Point-Cloud Object Detection without 3D Annotationcited by 81
  11. 2022Medical Image Understanding with Pretrained Vision Language Models: A Comprehensive Studycited by 66
  12. 2023Sentence-level Prompts Benefit Composed Image Retrievalcited by 59
38 more
  1. 2024TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language Negativescited by 57
  2. 2023Towards Language Models That Can See: Computer Vision Through the LENS of Natural Languagecited by 53
  3. 2024VHELM: A Holistic Evaluation of Vision Language Modelscited by 53
  4. 2023Context-I2W: Mapping Images to Context-dependent Words for Accurate Zero-Shot Composed Image Retrievalcited by 52
  5. 2024MedCoT: Medical Chain of Thought via Hierarchical Expertcited by 46
  6. 2025Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversioncited by 42
  7. 2022VL-BEiT: Generative Vision-Language Pretrainingcited by 39
  8. 2024One Perturbation is Enough: On Generating Universal Adversarial Perturbations against Vision-Language Pre-training Modelscited by 39
  9. 2023ConZIC: Controllable Zero-shot Image Captioning by Sampling-Based Polishingcited by 33
  10. 2024Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrievalcited by 31
  11. 2023CLIP4STR: A Simple Baseline for Scene Text Recognition with Pre-trained Vision-Language Modelcited by 30
  12. 2023ROME: Evaluating Pre-trained Vision-Language Models on Reasoning beyond Visual Common Sensecited by 27
  13. 2022I Can't Believe There's No Images! Learning Visual Tasks Using only Language Supervisioncited by 26
  14. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 24
  15. 2023Robust Visual Question Answering: Datasets, Methods, and Future Challengescited by 23
  16. 2024R-LLaVA: Improving Med-VQA Understanding through Visual Region of Interestcited by 21
  17. 2022Reassessing Evaluation Practices in Visual Question Answering: A Case Study on Out-of-Distribution Generalizationcited by 20
  18. 2023Meta Learning to Bridge Vision and Language Models for Multimodal Few-Shot Learningcited by 20
  19. 2022CPL: Counterfactual Prompt Learning for Vision and Language Modelscited by 18
  20. 2024Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillationcited by 18
  21. 2024Image2Struct: Benchmarking Structure Extraction for Vision-Language Modelscited by 18
  22. 2022Prototypical VoteNet for Few-Shot 3D Point Cloud Object Detectioncited by 14
  23. 2023UKnow: A Unified Knowledge Protocol with Multimodal Knowledge Graph Datasets for Reasoning and Vision-Language Pre-Trainingcited by 14
  24. 2023Towards General Purpose Medical AI: Continual Learning Medical Foundation Modelcited by 14
  25. 2023Symmetrical Linguistic Feature Distillation with CLIP for Scene Text Recognitioncited by 14
  26. 2024Vision-Language Model Fine-Tuning via Simple Parameter-Efficient Modificationcited by 14
  27. 2024Turbo: Informativity-Driven Acceleration Plug-In for Vision-Language Large Modelscited by 13
  28. 2024Advancing Myopia To Holism: Fully Contrastive Language-Image Pre-trainingcited by 13
  29. 2023In Search for a Generalizable Method for Source Free Domain Adaptationcited by 12
  30. 2024Interpreting and Analysing CLIP's Zero-Shot Image Classification via Mutual Knowledgecited by 12
  31. 2023Towards Medical Artificial General Intelligence via Knowledge-Enhanced Multimodal Pretrainingcited by 11
  32. 2023Turbo: Informativity-Driven Acceleration Plug-In for Vision-Language Modelscited by 11
  33. 2025Logits DeConfusion with CLIP for Few-Shot Learningcited by 11
  34. 2023CgT-GAN: CLIP-guided Text GAN for Image Captioningcited by 10
  35. 2024HELPER-X: A Unified Instructable Embodied Agent to Tackle Four Interactive Vision-Language Domains with Memory-Augmented Language Modelscited by 8
  36. 2022Multilingual Relation Classification via Efficient and Effective Promptingcited by 7
  37. 2023KAFA: Rethinking Image Ad Understanding with Knowledge-Augmented Feature Adaptation of Vision-Language Modelscited by 7
  38. 2023Explaining CLIP's performance disparities on data from blind/low vision userscited by 7

50 most cited of 113 All 113 (JSON)