mlsanityData to 7 Oct 2026

Proto-CLIP: Vision-Language Prototypical Network for Few-Shot Learning

arXiv 2307.03073 ↗First posted 6 Jul 2023cs.CV cs.ROcited by 12

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

We propose a novel framework for few-shot learning by leveraging large-scale vision-language models such as CLIP. Motivated by unimodal prototypical networks for few-shot learning, we introduce Proto-CLIP which utilizes image prototypes and text prototypes for few-shot learning. Specifically, Proto-CLIP adapts the image and text encoder embeddings from CLIP in a joint fashion using few-shot examples. The embeddings from the two encoders are used to compute the respective prototypes of image classes for classification. During adaptation, we propose aligning the image and text prototypes of the corresponding classes. Such alignment is beneficial for few-shot classification due to the reinforced contributions from both types of prototypes. Proto-CLIP has both training-free and fine-tuned variants. We demonstrate the effectiveness of our method by conducting experiments on benchmark datasets for few-shot learning, as well as in the real world for robot perception. The project page is available at https://irvlutd.github.io/Proto-CLIP

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 27 of its 39 references that are in the map. First posted on arXiv 6 Jul 2023.

Builds on

27 of 39 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022SuS-X: Training-Free Name-Only Transfer of Vision-Language Models28 Nov
  2. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification19 Jul
  3. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapters9 Oct
  4. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 23

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  6. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  7. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  8. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  9. 2016Matching Networks for One Shot Learningcited by 2,246
  10. 2013Describing Textures in the Wildcited by 2,110
  11. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  12. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
11 more
  1. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  2. 2019A Closer Look at Few-shot Classificationcited by 679
  3. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
  4. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
  5. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  6. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
  7. 2021Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenescited by 213
  8. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
  9. 2015Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocolscited by 98
  10. 2022Mean Shift Mask Transformer for Unseen Object Instance Segmentationcited by 19
  11. 2022FewSOL: A Dataset for Few-Shot Object Learning in Robotic Environmentscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 6; 2026: 3.

12 papers.

  1. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  2. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Drivingcited by 136
  3. 2025Can Masked Autoencoders Also Listen to Birds?cited by 26
  4. 2024Adapting Pre-Trained Vision Models for Novel Instance Detection and Segmentationcited by 17
  5. 2025Logits DeConfusion with CLIP for Few-Shot Learningcited by 11
  6. 2025Multi-Cache Enhanced Prototype Learning for Test-Time Generalization of Vision-Language Modelscited by 6
  7. 2025Improving Generalization of Language-Conditioned Robot Manipulationcited by 1
  8. 2026Unsupervised Data-Efficient Cross-Modal Retrieval with Global-Neighborhood Alignment Hashingcited by 1
  9. 2025Overcoming Labelled Data Scarcity for Defect Classification in Scanning Tunneling Microscopycited by 0
  10. 2025Attn-Adapter: Attention Is All You Need for Online Few-shot Learner of Vision-Language Modelcited by 0
  11. 2026Scalable Semantic Steering of Embedding Projectionscited by 0
  12. 2026Reprogramming Vision-Language Models via Structured Prompt Reparameterizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Prompting through Prototype: A Prototype-based Prompt Learning on Pretrained Vision-Language Models
  2. 2025Probabilistic Prototype Calibration of Vision-Language Models for Generalized Few-shot Semantic Segmentation
  3. 2025Dynamic Multimodal Prototype Learning in Vision-Language Models
  4. 2023Meta-Adapter: An Online Few-shot Learner for Vision-Language Model
  5. 2026Cross-Modal Prototype Alignment and Mixing for Training-Free Few-Shot Classification
  6. 2025Vocabulary-free few-shot learning for Vision-Language Models
  7. 2023ECO: Ensembling Context Optimization for Vision-Language Models
  8. 2025Can Synthetic Images Serve as Effective and Efficient Class Prototypes?