mlsanityData to 7 Oct 2026

Boosting Vision-Language Models with Transduction

arXiv 2406.01837 ↗First posted 3 Jun 2024cs.CVcited by 36

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Transduction is a powerful paradigm that leverages the structure of unlabeled data to boost predictive accuracy. We present TransCLIP, a novel and computationally efficient transductive approach designed for Vision-Language Models (VLMs). TransCLIP is applicable as a plug-and-play module on top of popular inductive zero- and few-shot models, consistently improving their performances. Our new objective function can be viewed as a regularized maximum-likelihood estimation, constrained by a KL divergence penalty that integrates the text-encoder knowledge and guides the transductive learning process. We further derive an iterative Block Majorize-Minimize (BMM) procedure for optimizing our objective, with guaranteed convergence and decoupled sample-assignment updates, yielding computationally efficient transduction for large-scale datasets. We report comprehensive evaluations, comparisons, and ablation studies that demonstrate: (i) Transduction can greatly enhance the generalization capabilities of inductive pretrained zero- and few-shot VLMs; (ii) TransCLIP substantially outperforms standard transductive few-shot learning methods relying solely on vision features, notably due to the KL-based language constraint.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 51 of its 75 references that are in the map. First posted on arXiv 3 Jun 2024.

Builds on

51 of 75 references.

Closest ancestors · 16

Same direction, newest first

  1. 2024Low-Rank Few-Shot Adaptation of Vision-Language Models28 May
  2. 2024On the test-time zero-shot generalization of vision-language models: Do we really need prompt learning?3 May
  3. 2023Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalization2 Nov
  4. 2023Distribution-Aware Prompt Tuning for Vision-Language Models6 Sep
  5. 2023Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuning11 Aug
  6. 2023Black Box Few-Shot Adaptation for Vision-Language models4 Apr
  7. 2023Visual-Language Prompt Tuning with Knowledge-guided Context Optimization23 Mar
  8. 2022Task Residual for Tuning Vision-Language Models18 Nov
  9. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models15 Sep
  10. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification19 Jul
  11. 2022Prompt-aligned Gradient for Prompt Tuning30 May
  12. 2022Prompt Distribution Learning6 May
4 more
  1. 2022Unsupervised Prompt Learning for Vision-Language Models7 Apr
  2. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  3. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapters9 Oct
  4. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  3. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  5. 2013Describing Textures in the Wildcited by 2,110
  6. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  7. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  8. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  9. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  10. 2019Natural Adversarial Examplescited by 1,401
  11. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  12. 2019How Can We Know What Language Models Know?cited by 953
23 more
  1. 2020Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptationcited by 844
  2. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  3. 2021Robust fine-tuning of zero-shot modelscited by 785
  4. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  5. 2019A Baseline for Few-Shot Image Classificationcited by 298
  6. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  7. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  8. 2020Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shiftcited by 235
  9. 2018Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learningcited by 208
  10. 2022CALIP: Zero-Shot Enhancement of CLIP with Parameter-free Attentioncited by 103
  11. 2019Prototype Rectification for Few-Shot Learningcited by 96
  12. 2020Laplacian Regularized Few-Shot Learningcited by 90
  13. 2024No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performancecited by 75
  14. 2020Leveraging the Feature Distribution in Transfer-based Few-Shot Learningcited by 72
  15. 2022LASP: Text-to-Text Optimization for Language-Aware Soft Prompting of Vision & Language Modelscited by 58
  16. 2024A Hard-to-Beat Baseline for Training-free CLIP-based Adaptationcited by 51
  17. 2020Iterative label cleaning for transductive and semi-supervised few-shot learningcited by 40
  18. 2024Transductive Zero-Shot and Few-Shot CLIPcited by 31
  19. 2022Realistic Evaluation of Transductive Few-Shot Learningcited by 26
  20. 2013An Information-Theoretic Analysis of Hard and Soft Assignment Methods for Clusteringcited by 17
  21. 2013Iteration Complexity Analysis of Block Coordinate Descent Methodscited by 17
  22. 2022Towards Practical Few-Shot Query Sets: Transductive Minimum Description Length Inferencecited by 9
  23. 2022TransBoost: Improving the Best ImageNet Performance using Deep Transductioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 13; 2026: 19.

36 papers.

Sort:

By area:

  1. 2025Realistic Test-Time Adaptation of Vision-Language Modelscited by 14
  2. 2025Conformal Prediction for Zero-Shot Modelscited by 14
  3. 2024textttBATCLIP: Bimodal Online Test-Time Adaptation for CLIPcited by 11
  4. 2024Enhancing Remote Sensing Vision-Language Models for Zero-Shot Scene Classificationcited by 8
  5. 2024Boosting Vision-Language Models for Histopathology Classification: Predict all at oncecited by 8
  6. 2025Trustworthy Few-Shot Transfer of Medical VLMs through Split Conformal Predictioncited by 8
  7. 2025Online Gaussian Test-Time Adaptation of Vision-Language Modelscited by 6
  8. 2025Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignmentcited by 5
  9. 2025Generate, Transduct, Adapt: Iterative Transduction with VLMscited by 2
  10. 2026LATA: Laplacian-Assisted Transductive Adaptation for Conformal Uncertainty in Medical VLMscited by 2
  11. 2026ConInfer: Context-Aware Inference for Training-Free Open-Vocabulary Remote Sensing Segmentationcited by 2
  12. 2026Whole-Slide Image Analysis under Realistic Few-Shot Annotation Protocolscited by 2
24 more
  1. 2024Physically Interpretable Probabilistic Domain Characterizationcited by 1
  2. 2025Enhancing Zero-Shot Image Recognition in Vision-Language Models through Human-like Concept Guidancecited by 1
  3. 2025Towards Fine-Grained Adaptation of CLIP via a Self-Trained Alignment Scorecited by 1
  4. 2025UniFGVC: Universal Training-Free Few-Shot Fine-Grained Vision Classification via Attribute-Aware Multimodal Retrievalcited by 1
  5. 2026Leveraging Prediction Entropy for Automatic Prompt Weighting in Zero-Shot Audio-Language Classificationcited by 1
  6. 2026ORION: ORthonormal Text Encoding for Universal VLM AdaptatIONcited by 1
  7. 2026Locally Consistent Transductive Information Maximization for Few-Shot Remote Sensing Scene Classificationcited by 1
  8. 2025Unbiased Max-Min Embedding Classification for Transductive Few-Shot Learning: Clustering and Classification Are All You Needcited by 0
  9. 2025Language-Aware Information Maximization for Transductive Few-Shot CLIPcited by 0
  10. 2025Cat: Post-Training Quantization Error Reduction via Cluster-based Affine Transformationcited by 0
  11. 2025GTMA: Dynamic Representation Optimization for OOD Vision-Language Modelscited by 0
  12. 2026Multi-Agent Cooperative Learning for Robust Vision-Language Alignment under OOD Conceptscited by 0
  13. 2026Retrieve and Segment: Are a Few Examples Enough to Bridge the Supervision Gap in Open-Vocabulary Segmentation?cited by 0
  14. 2026Semi-Supervised Few-Shot Adaptation of Vision-Language Modelscited by 0
  15. 2026Beyond Heuristic Prompting: A Concept-Guided Bayesian Framework for Zero-Shot Image Recognitioncited by 0
  16. 2026ACE-LoRA: Graph-Attentive Context Enhancement for Parameter-Efficient Adaptation of Medical Vision-Language Modelscited by 0
  17. 2026Training-Only Heterogeneous Image-Patch-Text Graph Supervision for Advancing Few-Shot Learning Adapterscited by 0
  18. 2026[CLS] is Not Enough: Multi-Label Recognition via Patch-Level Inference and Adaptive Aggregationcited by 0
  19. 2026MonoIR-RS: Infrared Remote Sensing Vision-Language Learning with CLIP and VLM Adaptationcited by 0
  20. 2026PRiSM: Prototype Regularization for Few-Shot VLMscited by 0
  21. 2026Dynamic Distribution-Aware Uncertainty Tracking in Vision-Language Representation Learningcited by 0
  22. 2026Learning to Adapt and Calibrate: Score Distribution Alignment for Few-Shot Uncertainty Prediction in Medical VLMscited by 0
  23. 2026Balanced Prompt Adaptation against Entropy-Induced Collapse for Test-Time Binary Segmentationcited by 0
  24. 2026Exploiting Spatial Structure for Transductive Few-Shot Classification of Whole-Slide Imagescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Generalizing vision-language models to novel domains: A comprehensive survey
  2. 2025Transferable Model-agnostic Vision-Language Model Adaptation for Efficient Weak-to-Strong Generalization
  3. 2023A Closer Look at the Few-Shot Adaptation of Large Vision-Language Models
  4. 2025LoRA-TTT: Low-Rank Test-Time Training for Vision-Language Models
  5. 2025TransPrune: Token Transition Pruning for Efficient Large Vision-Language Model
  6. 2026From Inference to Adaptation: A Unified Optimal Transport View of Vision Language Model
  7. 2023Robust Fine-Tuning of Vision-Language Models for Domain Generalization
  8. 2026TGV-KV: Text-Grounded KV Eviction for Vision-Language Models