mlsanityData to 7 Oct 2026

Personalizing Pre-trained Models

arXiv 2106.01499 ↗First posted 2 Jun 2021cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Self-supervised or weakly supervised models trained on large-scale datasets have shown sample-efficient transfer to diverse datasets in few-shot settings. We consider how upstream pretrained models can be leveraged for downstream few-shot, multilabel, and continual learning tasks. Our model CLIPPER (CLIP PERsonalized) uses image representations from CLIP, a large-scale image representation learning model trained using weak natural language supervision. We developed a technique, called Multi-label Weight Imprinting (MWI), for multi-label, continual, and few-shot learning, and CLIPPER uses MWI with image representations from CLIP. We evaluated CLIPPER on 10 single-label and 5 multi-label datasets. Our model shows robust and competitive performance, and we set new benchmarks for few-shot, multi-label, and continual learning. Our lightweight technique is also compute-efficient and enables privacy-preserving applications as the data is not sent to the upstream model for fine-tuning.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 53 of its 76 references that are in the map. First posted on arXiv 2 Jun 2021.

Builds on

53 of 76 references.

References · 53

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  9. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  10. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  11. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  12. 2017Prototypical Networks for Few-shot Learningcited by 3,001
38 more
  1. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  2. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  3. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  4. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  5. 2016Matching Networks for One Shot Learningcited by 2,246
  6. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  7. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  8. 2019Natural Adversarial Examplescited by 1,401
  9. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  10. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  11. 2018Experience Replay for Continual Learningcited by 1,074
  12. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  13. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  14. 2019A Closer Look at Few-shot Classificationcited by 679
  15. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  16. 2021ImageNet-21K Pretraining for the Massescited by 607
  17. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  18. 2018Do Better ImageNet Models Transfer Better?cited by 560
  19. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
  20. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
  21. 2017Age Progression/Regression by Conditional Adversarial Autoencodercited by 426
  22. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  23. 2017Meta Networkscited by 268
  24. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  25. 2020How Well Do Self-Supervised Models Transfer?cited by 213
  26. 2015Semi-supervised Sequence Learningcited by 198
  27. 2016CNN-RNN: A Unified Framework for Multi-label Image Classificationcited by 189
  28. 2019Boosting Few-Shot Visual Learning with Self-Supervisioncited by 140
  29. 2018Memory Efficient Experience Replay for Streaming Learningcited by 128
  30. 2021Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labelscited by 117
  31. 2018PCL: Proposal Cluster Learning for Weakly Supervised Object Detectioncited by 97
  32. 2019FairFace: Face Attribute Dataset for Balanced Race, Gender, and Agecited by 87
  33. 2016One-shot Learning with Memory-Augmented Neural Networkscited by 60
  34. 2020Pi-nets: Deep Polynomial Neural Networkscited by 39
  35. 2018Multi-Label Image Classification via Knowledge Distillation from Weakly-Supervised Detectioncited by 37
  36. 2019The iMaterialist Fashion Attribute Datasetcited by 37
  37. 2020Null-sampling for Interpretable and Fair Representationscited by 16
  38. 2019Direction Concentration Learning: Enhancing Congruency in Machine Learningcited by 5

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1; 2023: 1; 2024: 1; 2025: 1.

5 papers.

  1. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  2. 2022"This is my unicorn, Fluffy": Personalizing frozen vision-language representationscited by 95
  3. 2023Agile Modeling: From Concept to Classifier in Minutescited by 8
  4. 2021Pretrained Encoders are All You Needcited by 2
  5. 2025Robust Weight Imprinting: Insights from Neural Collapse and Proxy-Based Aggregationcited by 0