mlsanityData to 14 Oct 2026

Generate, Transduct, Adapt: Iterative Transduction with VLMs

arXiv 2501.06031 ↗First posted 10 Jan 2025cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Transductive zero-shot learning with vision-language models leverages image-image similarities within the dataset to achieve better classification accuracy compared to the inductive setting. However, there is little work that explores the structure of the language space in this context. We propose GTA-CLIP, a novel technique that incorporates supervision from language models for joint transduction in language and vision spaces. Our approach is iterative and consists of three steps: (i) incrementally exploring the attribute space by querying language models, (ii) an attribute-augmented transductive inference procedure, and (iii) fine-tuning the language and vision encoders based on inferred labels within the dataset. Through experiments with CLIP encoders, we demonstrate that GTA-CLIP, yields an average performance improvement of 8.6% and 3.7% across 12 datasets and 3 encoders, over CLIP and transductive CLIP respectively in the zero-shot setting. We also observe similar improvements in a few-shot setting. We present ablation studies that demonstrate the value of each step and visualize how the vision and language spaces evolve over iterations driven by the transductive learning. Code is released at https://github.com/cvl-umass/GTA-CLIP

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 47 of its 67 references that are in the map. First posted on arXiv 10 Jan 2025.

Builds on

47 of 67 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024Boosting Vision-Language Models with Transduction3 Jun
  2. 2024Label Propagation for Zero-shot Classification with Vision-Language Models5 Apr
  3. 2024Improved Zero-Shot Classification by Adapting VLMs with Text Descriptions4 Jan
  4. 2023Learning Concise and Descriptive Attributes for Visual Recognition7 Aug
  5. 2023Enhancing CLIP with GPT-4: Harnessing Visual Descriptions as Prompts21 Jul
  6. 2022Visual Classification via Description from Large Language Models13 Oct
  7. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  8. 2021A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Model29 Dec
  9. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling6 Nov
  10. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapters9 Oct
  11. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2013Describing Textures in the Wildcited by 2,125
  6. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
  7. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  8. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  9. 2021Robust fine-tuning of zero-shot modelscited by 797
  10. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 616
  11. 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 540
  12. 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 532
24 more
  1. 2021Image Segmentation Using Text and Image Promptscited by 512
  2. 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 510
  3. 2023Scaling Open-Vocabulary Object Detectioncited by 343
  4. 2022What does a platypus look like? Generating customized prompts for zero-shot image classificationcited by 267
  5. 2021A Survey on Deep Semi-supervised Learningcited by 251
  6. 2019Billion-scale semi-supervised learning for image classificationcited by 232
  7. 2022SegCLIP: Patch Aggregation with Learnable Centers for Open-Vocabulary Semantic Segmentationcited by 164
  8. 2021VL-LTR: Learning Class-wise Visual-Linguistic Representation for Long-Tailed Visual Recognitioncited by 65
  9. 2023AD-CLIP: Adapting Domains in Prompt Space Using CLIPcited by 62
  10. 2022I2MVFormer: Large Language Model Generated Multi-View Document Supervision for Zero-Shot Image Classificationcited by 54
  11. 2015Transductive Multi-view Zero-Shot Learningcited by 53
  12. 2023LaFTer: Label-Free Tuning of Zero-shot Classifier using Language and Unlabeled Image Collectionscited by 43
  13. 2021Domain Prompt Learning for Efficiently Adapting CLIP to Unseen Domainscited by 39
  14. 2019Transductive Zero-Shot Learning with Visual Structure Constraintcited by 36
  15. 2024Transductive Zero-Shot and Few-Shot CLIPcited by 32
  16. 2015Transductive Zero-Shot Action Recognition by Word-Vector Embeddingcited by 31
  17. 2017Zero-Shot Learning via Class-Conditioned Deep Generative Modelscited by 21
  18. 2024Evolving Interpretable Visual Classifiers with Large Language Modelscited by 18
  19. 2013Iteration Complexity Analysis of Block Coordinate Descent Methodscited by 17
  20. 2021Hardness Sampling for Self-Training Based Transductive Zero-Shot Learningcited by 12
  21. 2023GIST: Generating Image-Specific Text for Fine-grained Object Classificationcited by 8
  22. 2017Transductive Zero-Shot Learning with a Self-training dictionary approachcited by 7
  23. 2017Transductive Zero-Shot Learning with Adaptive Structural Embeddingcited by 7
  24. 2022An Iterative Co-Training Transductive Framework for Zero Shot Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Not All Birds Look The Same: Identity-Preserving Generation For Birdscited by 0
  2. 2026RealBirdID: Benchmarking Bird Species Identification in the Era of MLLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Language-Aware Information Maximization for Transductive Few-Shot CLIP
  2. 2024Zoom-shot: Fast and Efficient Unsupervised Zero-Shot Transfer of CLIP to Vision Encoders with Multimodal Loss
  3. 2023A Closer Look at the Few-Shot Adaptation of Large Vision-Language Models
  4. 2023Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Models
  5. 2026Language-Conditioned Visual Grounding with CLIP Multilingual
  6. 2025Conformal Prediction for Zero-Shot Models
  7. 2026PERL: Parameter Efficient Reasoning in CLIP Latent Space
  8. 2024RWKV-CLIP: A Robust Vision-Language Representation Learner