mlsanityData to 7 Oct 2026

Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinement

arXiv 2304.01195 ↗First posted 3 Apr 2023cs.CV cs.AI cs.MMcited by 84

Not on the map. Why?

Cited by 44 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The popularity of Contrastive Language-Image Pre-training (CLIP) has propelled its application to diverse downstream vision tasks. To improve its capacity on downstream tasks, few-shot learning has become a widely-adopted technique. However, existing methods either exhibit limited performance or suffer from excessive learnable parameters. In this paper, we propose APE, an Adaptive Prior rEfinement method for CLIP's pre-trained knowledge, which achieves superior accuracy with high computational efficiency. Via a prior refinement module, we analyze the inter-class disparity in the downstream data and decouple the domain-specific knowledge from the CLIP-extracted cache model. On top of that, we introduce two model variants, a training-free APE and a training-required APE-T. We explore the trilateral affinities between the test image, prior cache model, and textual representations, and only enable a lightweight category-residual module to be trained. For the average accuracy over 11 benchmarks, both APE and APE-T attain state-of-the-art and respectively outperform the second-best by +1.59% and +1.99% under 16 shots with x30 less learnable parameters.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). Builds on 68 of its 109 references that are in the map. First posted on arXiv 3 Apr 2023.

Builds on

68 of 109 references.

References · 68

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  9. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  10. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  11. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  12. 2013Describing Textures in the Wildcited by 2,110
38 more
  1. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  2. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  3. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  4. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  5. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  6. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
  7. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  8. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  9. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  10. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  11. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  12. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
  13. 2020What is the State of Neural Network Pruning?cited by 578
  14. 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 530
  15. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 510
  16. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
  17. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 434
  18. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 429
  19. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 424
  20. 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
  21. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Modelscited by 379
  22. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  23. 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 317
  24. 2022Scaling Language-Image Pre-training via Maskingcited by 317
  25. 2022What does a platypus look like? Generating customized prompts for zero-shot image classificationcited by 263
  26. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  27. 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 220
  28. 2022Frozen CLIP Models are Efficient Video Learnerscited by 181
  29. 2022DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detectioncited by 176
  30. 2023Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learnerscited by 152
  31. 2022Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoderscited by 143
  32. 2023Vision-Language Models as Success Detectorscited by 138
  33. 2021Multimodal Contrastive Training for Visual Representation Learningcited by 122
  34. 2022SuS-X: Training-Free Name-Only Transfer of Vision-Language Modelscited by 118
  35. 2022ReCo: Retrieve and Co-segment for Zero-shot Transfercited by 115
  36. 2022CALIP: Zero-Shot Enhancement of CLIP with Parameter-free Attentioncited by 103
  37. 2022Decorate the Newcomers: Visual Domain Prompt for Continual Test Time Adaptationcited by 98
  38. 2022Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Modelscited by 95

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 37; 2025: 24; 2026: 14.

84 papers.

Sort:

By area:

  1. 2023ImageBind-LLM: Multi-modality Instruction Tuningcited by 142
  2. 2023GraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graphcited by 69
  3. 2023Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentationcited by 54
  4. 2024A Hard-to-Beat Baseline for Training-free CLIP-based Adaptationcited by 51
  5. 2024Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Modelscited by 51
  6. 2023Gradient-based Parameter Selection for Efficient Fine-Tuningcited by 40
  7. 2024Diffusion Feedback Helps CLIP See Bettercited by 37
  8. 2023Meta-Adapter: An Online Few-shot Learner for Vision-Language Modelcited by 31
  9. 2024MoPE-CLIP: Structured Pruning for Efficient Vision-Language Models with Module-wise Pruning Error Metriccited by 30
  10. 2024Label Propagation for Zero-shot Classification with Vision-Language Modelscited by 28
  11. 2024AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportationcited by 27
  12. 2024Concept-Guided Prompt Learning for Generalization in Vision-Language Modelscited by 20
38 more
  1. 2024Rethinking Prior Information Generation with CLIP for Few-Shot Segmentationcited by 20
  2. 2024Selective Vision-Language Subspace Projection for Few-shot CLIPcited by 18
  3. 2024BoostAdapter: Improving Vision-Language Test-Time Adaptation via Regional Bootstrappingcited by 17
  4. 2025ProKeR: A Kernel Perspective on Few-Shot Adaptation of Large Vision-Language Modelscited by 15
  5. 2025Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Modelcited by 15
  6. 2024Local-Prompt: Extensible Local Prompts for Few-Shot Out-of-Distribution Detectioncited by 14
  7. 2025Learning to Adapt Frozen CLIP for Few-Shot Test-Time Domain Adaptationcited by 14
  8. 2023FD-Align: Feature Discrimination Alignment for Fine-tuning Pre-Trained Models in Few-Shot Learningcited by 12
  9. 2025ProAPO: Progressively Automatic Prompt Optimization for Visual Classificationcited by 12
  10. 2023Towards Top-Down Reasoning: An Explainable Multi-Agent Approach for Visual Question Answeringcited by 11
  11. 2025Logits DeConfusion with CLIP for Few-Shot Learningcited by 11
  12. 2024Boosting Few-Shot Learning via Attentive Feature Regularizationcited by 10
  13. 2024What Do You See? Enhancing Zero-Shot Image Classification with Multimodal Large Language Modelscited by 10
  14. 2024ComKD-CLIP: Comprehensive Knowledge Distillation for Contrastive Language-Image Pre-traning Modelcited by 9
  15. 2024Enhancing Vision-Language Few-Shot Adaptation with Negative Learningcited by 8
  16. 2024FAST: A Dual-tier Few-Shot Learning Paradigm for Whole Slide Image Classificationcited by 8
  17. 2024Efficient and Context-Aware Label Propagation for Zero-/Few-Shot Training-Free Adaptation of Vision-Language Modelcited by 8
  18. 2024ZoRI: Towards Discriminative Zero-Shot Remote Sensing Instance Segmentationcited by 7
  19. 2024Text and Image Are Mutually Beneficial: Enhancing Training-Free Few-Shot Classification with CLIPcited by 6
  20. 2025Generalizing vision-language models to novel domains: A comprehensive surveycited by 6
  21. 2024Weakly-Supervised 3D Scene Graph Generation via Visual-Linguistic Assisted Pseudo-labelingcited by 5
  22. 2024Rethinking Visual Content Refinement in Low-Shot CLIP Adaptationcited by 5
  23. 2024DPA: Dual Prototypes Alignment for Unsupervised Adaptation of Vision-Language Modelscited by 5
  24. 2024TransAgent: Transfer Vision-Language Foundation Models with Heterogeneous Agent Collaborationcited by 5
  25. 2025Multimodal Conditional Information Bottleneck for Generalizable AI-Generated Image Detectioncited by 5
  26. 2025Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignmentcited by 5
  27. 2025Hierarchical Semantic Alignment for Image Clusteringcited by 5
  28. 2024DiffCLIP: Few-shot Language-driven Multimodal Classifiercited by 4
  29. 2025FLOSS: Free Lunch in Open-vocabulary Semantic Segmentationcited by 4
  30. 2024Conditional Prototype Rectification Prompt Learningcited by 3
  31. 2024NODE-Adapter: Neural Ordinary Differential Equations for Better Vision-Language Reasoningcited by 3
  32. 2024HeGraphAdapter: Tuning Multi-Modal Vision-Language Models with Heterogeneous Graph Adaptercited by 3
  33. 2024LatteCLIP: Unsupervised CLIP Fine-Tuning via LMM-Synthetic Textscited by 3
  34. 2024Calibrated Cache Model for Few-Shot Vision-Language Model Adaptationcited by 3
  35. 2024Context-Aware Multimodal Pretrainingcited by 3
  36. 2025Compositional Caching for Training-free Open-vocabulary Attribute Detectioncited by 3
  37. 2025Few-Shot Vision-Language Action-Incremental Policy Learningcited by 3
  38. 2025SeMoBridge: Semantic Modality Bridge for Efficient Few-Shot Adaptation of CLIPcited by 3

50 most cited of 84 All 84 (JSON)