mlsanityData to 7 Oct 2026

Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification

arXiv 2207.09519 ↗First posted 19 Jul 2022cs.CV cs.AI cs.CLcited by 355

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Contrastive Vision-Language Pre-training, known as CLIP, has provided a new paradigm for learning visual representations using large-scale image-text pairs. It shows impressive performance on downstream tasks by zero-shot knowledge transfer. To further enhance CLIP's adaption capability, existing methods proposed to fine-tune additional learnable modules, which significantly improves the few-shot performance but introduces extra training time and computational resources. In this paper, we propose a training-free adaption method for CLIP to conduct few-shot classification, termed as Tip-Adapter, which not only inherits the training-free advantage of zero-shot CLIP but also performs comparably to those training-required approaches. Tip-Adapter constructs the adapter via a key-value cache model from the few-shot training set, and updates the prior knowledge encoded in CLIP by feature retrieval. On top of that, the performance of Tip-Adapter can be further boosted to be state-of-the-art on ImageNet by fine-tuning the cache model for 10× fewer epochs than existing methods, which is both effective and efficient. We conduct extensive experiments of few-shot classification on 11 datasets to demonstrate the superiority of our proposed methods. Code is released at https://github.com/gaopengcuhk/Tip-Adapter.

Key facts. Cited by 355 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 57 of its 79 references that are in the map. First posted on arXiv 19 Jul 2022.

Builds on

57 of 79 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapters9 Oct
  2. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 55

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  6. 2020Language Models are Few-Shot Learnerscited by 32,199
  7. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  8. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  9. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  10. 2020End-to-End Object Detection with Transformerscited by 7,384
  11. 2017Mask R-CNNcited by 7,259
  12. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
38 more
  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  5. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  7. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  8. 2016Pointer Sentinel Mixture Modelscited by 2,992
  9. 2015VQA: Visual Question Answeringcited by 2,489
  10. 2017Billion-scale similarity search with GPUscited by 2,477
  11. 2016Matching Networks for One Shot Learningcited by 2,246
  12. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  13. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  14. 2013Describing Textures in the Wildcited by 2,110
  15. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  16. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  17. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  18. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  19. 2019Natural Adversarial Examplescited by 1,401
  20. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  21. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  22. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  23. 2021Robust fine-tuning of zero-shot modelscited by 785
  24. 2019A Closer Look at Few-shot Classificationcited by 679
  25. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
  26. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
  27. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 427
  28. 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
  29. 2019A Baseline for Few-Shot Image Classificationcited by 298
  30. 2019Attention on Attention for Image Captioningcited by 289
  31. 2018Bilinear Attention Networkscited by 277
  32. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 263
  33. 2016Image Captioning with Semantic Attentioncited by 260
  34. 2021Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learningcited by 112
  35. 2020End-to-End Object Detection with Adaptive Clustering Transformercited by 109
  36. 2022ConvMAE: Masked Convolution Meets Masked Autoencoderscited by 105
  37. 2022Multitask AET with Orthogonal Tangent Regularity for Dark Object Detectioncited by 59
  38. 2017Unbounded cache model for online language modeling with open vocabularycited by 37

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 59; 2024: 97; 2025: 116; 2026: 76.

355 papers.

Sort:

By area:

  1. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  2. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  3. 2023Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Followingcited by 192
  4. 2023Raising the Bar of AI-generated Image Detection with CLIPcited by 163
  5. 2024Efficient Test-Time Adaptation of Vision-Language Modelscited by 138
  6. 2022Task Residual for Tuning Vision-Language Modelscited by 122
  7. 2022SuS-X: Training-Free Name-Only Transfer of Vision-Language Modelscited by 118
  8. 2023Controlling Vision-Language Models for Multi-Task Image Restorationcited by 108
  9. 2024Low-Rank Few-Shot Adaptation of Vision-Language Modelscited by 103
  10. 2023Learning without Forgetting for Vision-Language Modelscited by 79
  11. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  12. 2025AA-CLIP: Enhancing Zero-shot Anomaly Detection via Anomaly-Aware CLIPcited by 72
38 more
  1. 2023ViewRefer: Grasp the Multi-view Knowledge for 3D Visual Grounding with GPT and Prototype Guidancecited by 70
  2. 2024Tracking Meets LoRA: Faster Training, Larger Model, Stronger Performancecited by 70
  3. 2023GraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graphcited by 69
  4. 2024On the test-time zero-shot generalization of vision-language models: Do we really need prompt learning?cited by 65
  5. 2024LP++: A Surprisingly Strong Linear Probe for Few-Shot CLIPcited by 60
  6. 2023PromptStyler: Prompt-driven Style Generation for Source-free Domain Generalizationcited by 59
  7. 2023DePT: Decoupled Prompt Tuningcited by 58
  8. 2023Diversity-Aware Meta Visual Promptingcited by 57
  9. 2023Joint-MAE: 2D-3D Joint Masked Autoencoders for 3D Point Cloud Pre-trainingcited by 52
  10. 2022Texts as Images in Prompt Tuning for Multi-Label Image Recognitioncited by 51
  11. 2023Improving Zero-Shot Generalization for CLIP with Synthesized Promptscited by 51
  12. 2024A Hard-to-Beat Baseline for Training-free CLIP-based Adaptationcited by 51
  13. 2023Training on Thin Air: Improve Image Classification with Generated Datacited by 50
  14. 2023Enhancing CLIP with GPT-4: Harnessing Visual Descriptions as Promptscited by 48
  15. 2023VIP5: Towards Multimodal Foundation Models for Recommendationcited by 45
  16. 2023FLIP: Cross-domain Face Anti-spoofing with Language Guidancecited by 45
  17. 2023Point-PEFT: Parameter-Efficient Fine-Tuning for 3D Pre-trained Modelscited by 42
  18. 2022Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Modelscited by 40
  19. 2023Improving neural network representations using human similarity judgmentscited by 38
  20. 2023How Does Fine-Tuning Impact Out-of-Distribution Detection for Vision-Language Models?cited by 38
  21. 2024Knowledge Composition using Task Vectors with Learned Anisotropic Scalingcited by 37
  22. 2024Diffusion Feedback Helps CLIP See Bettercited by 37
  23. 2023GL-MCM: Global and Local Maximum Concept Matching for Zero-Shot Out-of-Distribution Detectioncited by 36
  24. 2024Knowledge-Enhanced Dual-stream Zero-shot Composed Image Retrievalcited by 36
  25. 2024Boosting Vision-Language Models with Transductioncited by 36
  26. 2024Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Surveycited by 36
  27. 2024Dual Prototype Evolving for Test-Time Generalization of Vision-Language Modelscited by 36
  28. 2022SgVA-CLIP: Semantic-guided Visual Adapting of Vision-Language Models for Few-shot Image Classificationcited by 34
  29. 2024A Survey on Data Augmentation in Large Model Eracited by 31
  30. 2024GalLoP: Learning Global and Local Prompts for Vision-Language Modelscited by 29
  31. 2023Intra-Modal Proxy Learning for Zero-Shot Visual Categorization with CLIPcited by 27
  32. 2024AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportationcited by 27
  33. 2025Localized Concept Erasure for Text-to-Image Diffusion Models Using Training-Free Gated Low-Rank Adaptationcited by 27
  34. 2024Enhancing Zero-Shot Vision Models by Label-Free Prompt Distribution Learning and Bias Correctingcited by 26
  35. 2024CLAP4CLIP: Continual Learning with Probabilistic Finetuning for Vision-Language Modelscited by 25
  36. 2023Defense-Prefix for Preventing Typographic Attacks on CLIPcited by 24
  37. 2023Parameter-Efficient Transfer Learning for Remote Sensing Image-Text Retrievalcited by 24
  38. 2023Improving Compositional Text-to-image Generation with Large Vision-Language Modelscited by 24

50 most cited of 355 All 355 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling
  2. 2023Meta-Adapter: An Online Few-shot Learner for Vision-Language Model
  3. 2025ProKeR: A Kernel Perspective on Few-Shot Adaptation of Large Vision-Language Models
  4. 2025Attn-Adapter: Attention Is All You Need for Online Few-shot Learner of Vision-Language Model
  5. 2024Multi-Modal Adapter for Vision-Language Models
  6. 2025CLIP-SVD: Efficient and Interpretable Vision-Language Adaptation via Singular Values
  7. 2024CLIPArTT: Adaptation of CLIP to New Domains at Test Time
  8. 2023CLIP meets Model Zoo Experts: Pseudo-Supervision for Visual Enhancement