# Boosting Vision-Language Models with Transduction

arXiv: https://arxiv.org/abs/2406.01837 · HTML: https://mlsanity.com/paper/2406.01837/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 51 of its 75 references that are in the map.
- First posted on arXiv 3 Jun 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 30 of 51)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [Learning Robust Global Representations by Penalizing Local Predictive Power](https://mlsanity.com/paper/1905.13549/) (2019)
- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [Robust fine-tuning of zero-shot models](https://mlsanity.com/paper/2109.01903/) (2021)
- [MaPLe: Multi-modal Prompt Learning](https://mlsanity.com/paper/2210.03117/) (2022)
- [Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models](https://mlsanity.com/paper/2209.07511/) (2022)
- [Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification](https://mlsanity.com/paper/2207.09519/) (2022)
- [Prompt-aligned Gradient for Prompt Tuning](https://mlsanity.com/paper/2205.14865/) (2022)
- [Visual-Language Prompt Tuning with Knowledge-guided Context Optimization](https://mlsanity.com/paper/2303.13283/) (2023)
- [A Baseline for Few-Shot Image Classification](https://mlsanity.com/paper/1909.02729/) (2019)
- [Factual Probing Is [MASK]: Learning vs. Learning to Recall](https://mlsanity.com/paper/2104.05240/) (2021)
- [Self-regulating Prompts: Foundational Model Adaptation without Forgetting](https://mlsanity.com/paper/2307.06948/) (2023)
- [Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift](https://mlsanity.com/paper/2006.10963/) (2020)
- [Prompt Distribution Learning](https://mlsanity.com/paper/2205.03340/) (2022)
- [Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning](https://mlsanity.com/paper/1805.10002/) (2018)
- [Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuning](https://mlsanity.com/paper/2308.06038/) (2023)

## Built on by (most cited first, 30 of 36)

- [Conformal Prediction for Zero-Shot Models](https://mlsanity.com/paper/2505.24693/) (2025)
- [Realistic Test-Time Adaptation of Vision-Language Models](https://mlsanity.com/paper/2501.03729/) (2025)
- [textttBATCLIP: Bimodal Online Test-Time Adaptation for CLIP](https://mlsanity.com/paper/2412.02837/) (2024)
- [Enhancing Remote Sensing Vision-Language Models for Zero-Shot Scene Classification](https://mlsanity.com/paper/2409.00698/) (2024)
- [Boosting Vision-Language Models for Histopathology Classification: Predict all at once](https://mlsanity.com/paper/2409.01883/) (2024)
- [Trustworthy Few-Shot Transfer of Medical VLMs through Split Conformal Prediction](https://mlsanity.com/paper/2506.17503/) (2025)
- [Online Gaussian Test-Time Adaptation of Vision-Language Models](https://mlsanity.com/paper/2501.04352/) (2025)
- [Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignment](https://mlsanity.com/paper/2508.15568/) (2025)
- [ConInfer: Context-Aware Inference for Training-Free Open-Vocabulary Remote Sensing Segmentation](https://mlsanity.com/paper/2603.29271/) (2026)
- [Generate, Transduct, Adapt: Iterative Transduction with VLMs](https://mlsanity.com/paper/2501.06031/) (2025)
- [LATA: Laplacian-Assisted Transductive Adaptation for Conformal Uncertainty in Medical VLMs](https://mlsanity.com/paper/2602.17535/) (2026)
- [Whole-Slide Image Analysis under Realistic Few-Shot Annotation Protocols](https://mlsanity.com/paper/2608.30420/) (2026)
- [Physically Interpretable Probabilistic Domain Characterization](https://mlsanity.com/paper/2411.14827/) (2024)
- [Enhancing Zero-Shot Image Recognition in Vision-Language Models through Human-like Concept Guidance](https://mlsanity.com/paper/2503.15886/) (2025)
- [Towards Fine-Grained Adaptation of CLIP via a Self-Trained Alignment Score](https://mlsanity.com/paper/2507.09615/) (2025)
- [UniFGVC: Universal Training-Free Few-Shot Fine-Grained Vision Classification via Attribute-Aware Multimodal Retrieval](https://mlsanity.com/paper/2508.04136/) (2025)
- [Leveraging Prediction Entropy for Automatic Prompt Weighting in Zero-Shot Audio-Language Classification](https://mlsanity.com/paper/2601.05011/) (2026)
- [ORION: ORthonormal Text Encoding for Universal VLM AdaptatION](https://mlsanity.com/paper/2602.19530/) (2026)
- [ACE-LoRA: Graph-Attentive Context Enhancement for Parameter-Efficient Adaptation of Medical Vision-Language Models](https://mlsanity.com/paper/2603.17079/) (2026)
- [Locally Consistent Transductive Information Maximization for Few-Shot Remote Sensing Scene Classification](https://mlsanity.com/paper/2607.29192/) (2026)
- [Unbiased Max-Min Embedding Classification for Transductive Few-Shot Learning: Clustering and Classification Are All You Need](https://mlsanity.com/paper/2503.22193/) (2025)
- [Language-Aware Information Maximization for Transductive Few-Shot CLIP](https://mlsanity.com/paper/2509.00305/) (2025)
- [Cat: Post-Training Quantization Error Reduction via Cluster-based Affine Transformation](https://mlsanity.com/paper/2509.26277/) (2025)
- [GTMA: Dynamic Representation Optimization for OOD Vision-Language Models](https://mlsanity.com/paper/2512.18504/) (2025)
- [Multi-Agent Cooperative Learning for Robust Vision-Language Alignment under OOD Concepts](https://mlsanity.com/paper/2601.09746/) (2026)
- [Retrieve and Segment: Are a Few Examples Enough to Bridge the Supervision Gap in Open-Vocabulary Segmentation?](https://mlsanity.com/paper/2602.23339/) (2026)
- [Semi-Supervised Few-Shot Adaptation of Vision-Language Models](https://mlsanity.com/paper/2603.02959/) (2026)
- [Beyond Heuristic Prompting: A Concept-Guided Bayesian Framework for Zero-Shot Image Recognition](https://mlsanity.com/paper/2603.07911/) (2026)
- [Training-Only Heterogeneous Image-Patch-Text Graph Supervision for Advancing Few-Shot Learning Adapters](https://mlsanity.com/paper/2603.18101/) (2026)
- [[CLS] is Not Enough: Multi-Label Recognition via Patch-Level Inference and Adaptive Aggregation](https://mlsanity.com/paper/2605.25821/) (2026)

## Abstract (arXiv)

Transduction is a powerful paradigm that leverages the structure of unlabeled data to boost predictive accuracy. We present TransCLIP, a novel and computationally efficient transductive approach designed for Vision-Language Models (VLMs). TransCLIP is applicable as a plug-and-play module on top of popular inductive zero- and few-shot models, consistently improving their performances. Our new objective function can be viewed as a regularized maximum-likelihood estimation, constrained by a KL divergence penalty that integrates the text-encoder knowledge and guides the transductive learning process. We further derive an iterative Block Majorize-Minimize (BMM) procedure for optimizing our objective, with guaranteed convergence and decoupled sample-assignment updates, yielding computationally efficient transduction for large-scale datasets. We report comprehensive evaluations, comparisons, and ablation studies that demonstrate: (i) Transduction can greatly enhance the generalization capabilities of inductive pretrained zero- and few-shot VLMs; (ii) TransCLIP substantially outperforms standard transductive few-shot learning methods relying solely on vision features, notably due to the KL-based language constraint.