mlsanityData to 7 Oct 2026

Label Propagation for Zero-shot Classification with Vision-Language Models

arXiv 2404.04072 ↗First posted 5 Apr 2024cs.CV cs.LGcited by 28

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Vision-Language Models (VLMs) have demonstrated impressive performance on zero-shot classification, i.e. classification when provided merely with a list of class names. In this paper, we tackle the case of zero-shot classification in the presence of unlabeled data. We leverage the graph structure of the unlabeled data and introduce ZLaP, a method based on label propagation (LP) that utilizes geodesic distances for classification. We tailor LP to graphs containing both text and image features and further propose an efficient method for performing inductive inference based on a dual solution and a sparsification step. We perform extensive experiments to evaluate the effectiveness of our method on 14 common datasets and show that ZLaP outperforms the latest related works. Code: https://github.com/vladan-stojnic/ZLaP

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 47 references that are in the map. First posted on arXiv 5 Apr 2024.

Builds on

34 of 47 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023Intra-Modal Proxy Learning for Zero-Shot Visual Categorization with CLIP30 Oct
  2. 2023ReCLIP: Refine Contrastive Language Image Pre-Training with Source Free Domain Adaptation4 Aug
  3. 2023Waffling around for Performance: Visual Classification with Random Words and Broad Concepts12 Jun
  4. 2023Test-Time Distribution Normalization for Contrastively Learned Vision-language Models22 Feb
  5. 2022SuS-X: Training-Free Name-Only Transfer of Vision-Language Models28 Nov
  6. 2022Visual Classification via Description from Large Language Models13 Oct
  7. 2022SVL-Adapter: Self-Supervised Adapter for Vision-Language Pretrained Models7 Oct
  8. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models15 Sep
  9. 2022Unsupervised Prompt Learning for Vision-Language Models7 Apr
  10. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 24

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  5. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  6. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  7. 2013Describing Textures in the Wildcited by 2,110
  8. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  9. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  10. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  11. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  12. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
12 more
  1. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  2. 2019Label Propagation for Deep Semi-supervised Learningcited by 287
  3. 2022What does a platypus look like? Generating customized prompts for zero-shot image classificationcited by 263
  4. 2023Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learnerscited by 152
  5. 2022CALIP: Zero-Shot Enhancement of CLIP with Parameter-free Attentioncited by 103
  6. 2024EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameterscited by 99
  7. 2023Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinementcited by 84
  8. 2023CHiLS: Zero-Shot Image Classification with Hierarchical Label Setscited by 79
  9. 2016Efficient Diffusion on Region Manifolds: Recovering Small Objects with Compact CNN Representationscited by 49
  10. 2016Fast, Exact and Multi-Scale Inference for Semantic Image Segmentation with Deep Gaussian CRFscited by 27
  11. 2017Fast Spectral Ranking for Similarity Searchcited by 12
  12. 2023Neural Priming for Sample-Efficient Adaptationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 16; 2026: 6.

28 papers.

Sort:

By area:

  1. 2025LPOSS: Label Propagation Over Patches and Pixels for Open-vocabulary Semantic Segmentationcited by 21
  2. 2025Realistic Test-Time Adaptation of Vision-Language Modelscited by 14
  3. 2024textttBATCLIP: Bimodal Online Test-Time Adaptation for CLIPcited by 11
  4. 2024Efficient and Context-Aware Label Propagation for Zero-/Few-Shot Training-Free Adaptation of Vision-Language Modelcited by 8
  5. 2024Tuning-free Universally-Supervised Semantic Segmentationcited by 6
  6. 2025Online Gaussian Test-Time Adaptation of Vision-Language Modelscited by 6
  7. 2024On the Element-Wise Representation and Reasoning in Zero-Shot Image Recognition: A Systematic Surveycited by 5
  8. 2025Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignmentcited by 5
  9. 2025FLOSS: Free Lunch in Open-vocabulary Semantic Segmentationcited by 4
  10. 2024MiNT: Multi-Network Training for Transfer Learning on Temporal Graphscited by 2
  11. 2025Generate, Transduct, Adapt: Iterative Transduction with VLMscited by 2
  12. 2025An Explainable Deep Neural Network with Frequency-Aware Channel and Spatial Refinement for Flood Prediction in Sustainable Citiescited by 2
16 more
  1. 2026Whole-Slide Image Analysis under Realistic Few-Shot Annotation Protocolscited by 2
  2. 2025PVLM: Parsing-Aware Vision Language Model with Dynamic Contrastive Learning for Zero-Shot Deepfake Attributioncited by 1
  3. 2025Beyond Graph Model: Reliable VLM Fine-Tuning via Random Graph Adaptercited by 1
  4. 2025Slot Attention-based Feature Filtering for Few-Shot Learningcited by 1
  5. 2025Zero-Shot Multi-Spectral Learning: Reimagining a Generalist Multimodal Gemini 2.5 Model for Remote Sensing Applicationscited by 1
  6. 2026ORION: ORthonormal Text Encoding for Universal VLM AdaptatIONcited by 1
  7. 2024Domain-invariant Representation Learning via Segment Anything Model for Blood Cell Classificationcited by 0
  8. 2025Exploiting Vision Language Model for Training-Free 3D Point Cloud OOD Detection via Graph Score Propagationcited by 0
  9. 2025Stochastic-based Patch Filtering for Few-Shot Learningcited by 0
  10. 2025Category-level Text-to-Image Retrieval Improved: Bridging the Domain Gap with Diffusion Models and Vision Encoderscited by 0
  11. 2025Language-Aware Information Maximization for Transductive Few-Shot CLIPcited by 0
  12. 2025Consolidating Diffusion-Generated Video Detection with Unified Multimodal Forgery Learningcited by 0
  13. 2026Structure-aware Prompt Adaptation from Seen to Unseen for Open-Vocabulary Compositional Zero-Shot Learningcited by 0
  14. 2026Unlocking Multi-Spectral Data for Multi-Modal Models with Guided Inputs and Chain-of-Thought Reasoningcited by 0
  15. 2026Geometry-Preserving Unsupervised Alignment for Heterogeneous Foundation Modelscited by 0
  16. 2026Learning Deep Modality-Shared Self-Expressiveness for Image Clustering with Textual Informationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Enabling Small Models for Zero-Shot Selection and Reuse through Model Label Learning
  2. 2024LLM meets Vision-Language Models for Zero-Shot One-Class Classification
  3. 2025No Labels Needed: Zero-Shot Image Classification with Collaborative Self-Learning
  4. 2024Improved Zero-Shot Classification by Adapting VLMs with Text Descriptions
  5. 2025Interpretable Zero-Shot Learning with Locally-Aligned Vision-Language Model
  6. 2026Will It Zero-Shot?: Predicting Zero-Shot Classification Performance For Arbitrary Queries
  7. 2025Super-class guided Transformer for Zero-Shot Attribute Classification
  8. 2024Embracing Diversity: Interpretable Zero-shot classification beyond one vector per class