mlsanityData to 7 Oct 2026

Neural Priming for Sample-Efficient Adaptation

arXiv 2306.10191 ↗First posted 16 Jun 2023cs.LG cs.AI cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

We propose Neural Priming, a technique for adapting large pretrained models to distribution shifts and downstream tasks given few or no labeled examples. Presented with class names or unlabeled test samples, Neural Priming enables the model to recall and conditions its parameters on relevant data seen throughout pretraining, thereby priming it for the test distribution. Neural Priming can be performed at test time, even for pretraining datasets as large as LAION-2B. Performing lightweight updates on the recalled data significantly improves accuracy across a variety of distribution shift and transfer learning benchmarks. Concretely, in the zero-shot setting, we see a 2.45% improvement in accuracy on ImageNet and 3.81% accuracy improvement on average across standard transfer learning benchmarks. Further, using Neural Priming at inference to adapt to distribution shift, we see a 1.41% accuracy improvement on ImageNetV2. These results demonstrate the effectiveness of Neural Priming in addressing the challenge of limited labeled data and changing distributions. Code is available at github.com/RAIVNLab/neural-priming.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 36 of its 54 references that are in the map. First posted on arXiv 16 Jun 2023.

Builds on

36 of 54 references.

References · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  5. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  6. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  7. 2017Billion-scale similarity search with GPUscited by 2,477
  8. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  9. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  10. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  11. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  12. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
24 more
  1. 2019Natural Adversarial Examplescited by 1,401
  2. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  3. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  4. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  5. 2021Robust fine-tuning of zero-shot modelscited by 785
  6. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  7. 2020Measuring Robustness to Natural Distribution Shifts in Image Classificationcited by 452
  8. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  9. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Modelscited by 379
  10. 2018Rethinking ImageNet Pre-trainingcited by 376
  11. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classificationcited by 355
  12. 2022Visual Classification via Description from Large Language Modelscited by 331
  13. 2022What does a platypus look like? Generating customized prompts for zero-shot image classificationcited by 263
  14. 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 248
  15. 2021CLIP2Video: Mastering Video-Text Retrieval via Image CLIPcited by 223
  16. 2022Test-Time Training with Masked Autoencoderscited by 208
  17. 2022Surgical Fine-Tuning Improves Adaptation to Distribution Shiftscited by 205
  18. 2018Task-Agnostic Meta-Learning for Few-shot Learningcited by 151
  19. 2022Finetune like you pretrain: Improved finetuning of zero-shot vision modelscited by 149
  20. 2022Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP)cited by 137
  21. 2013Learning by Transductioncited by 120
  22. 2022SuS-X: Training-Free Name-Only Transfer of Vision-Language Modelscited by 118
  23. 2023Learning Customized Visual Models with Retrieval-Augmented Knowledgecited by 58
  24. 2023AdANNS: A Framework for Adaptive Semantic Searchcited by 2

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 5; 2025: 5.

12 papers.

  1. 2024The Neglected Tails in Vision-Language Modelscited by 63
  2. 2024Label Propagation for Zero-shot Classification with Vision-Language Modelscited by 28
  3. 2024The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Bettercited by 19
  4. 2024Few-Shot Recognition via Stage-Wise Retrieval-Augmented Finetuningcited by 15
  5. 2023Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Modelscited by 14
  6. 2023From Categories to Classifiers: Name-Only Continual Learning by Exploring the Webcited by 5
  7. 2024COBRA: COmBinatorial Retrieval Augmentation for Few-Shot Adaptationcited by 2
  8. 2025Active Learning via Vision-Language Model Adaptation with Open Datacited by 2
  9. 2025Enabling Validation for Robust Few-Shot Recognitioncited by 2
  10. 2025Domain Generalization in-the-Wild: Disentangling Classification from Domain-Aware Representationscited by 0
  11. 2025Solving Semi-Supervised Few-Shot Learning from an Auto-Annotation Perspectivecited by 0
  12. 2025D3G: Diverse Demographic Data Generation Increases Zero-Shot Image Classification Accuracy within Multimodal Modelscited by 0