# Transductive Information Maximization For Few-Shot Learning

arXiv: https://arxiv.org/abs/2008.11297 · HTML: https://mlsanity.com/paper/2008.11297/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Aug 2020.

## Built on by (most cited first, 30 of 45)

- [Realistic Evaluation of Transductive Few-Shot Learning](https://mlsanity.com/paper/2204.11181/) (2022)
- [Binocular Mutual Learning for Improving Few-shot Classification](https://mlsanity.com/paper/2108.12104/) (2021)
- [Graph Few-shot Learning with Task-specific Structures](https://mlsanity.com/paper/2210.12130/) (2022)
- [Matching Feature Sets for Few-Shot Image Classification](https://mlsanity.com/paper/2204.00949/) (2022)
- [ClusT3: Information Invariant Test-Time Training](https://mlsanity.com/paper/2310.12345/) (2023)
- [FHIST: A Benchmark for Few-shot Classification of Histological Images](https://mlsanity.com/paper/2206.00092/) (2022)
- [Self-Supervision Can Be a Good Few-Shot Learner](https://mlsanity.com/paper/2207.09176/) (2022)
- [Few-Shot Learning with Class Imbalance](https://mlsanity.com/paper/2101.02523/) (2021)
- [EASY: Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredients](https://mlsanity.com/paper/2201.09699/) (2022)
- [POODLE: Improving Few-shot Learning via Penalizing Out-of-Distribution Samples](https://mlsanity.com/paper/2206.04679/) (2022)
- [Conformal Prediction for Zero-Shot Models](https://mlsanity.com/paper/2505.24693/) (2025)
- [Squeezing Backbone Feature Distributions to the Max for Efficient Few-Shot Learning](https://mlsanity.com/paper/2110.09446/) (2021)
- [MetaNODE: Prototype Optimization as a Neural ODE for Few-Shot Learning](https://mlsanity.com/paper/2103.14341/) (2021)
- [Hubs and Hyperspheres: Reducing Hubness and Improving Transductive Few-shot Learning with Hyperspherical Embeddings](https://mlsanity.com/paper/2303.09352/) (2023)
- [Open-Set Likelihood Maximization for Few-Shot Learning](https://mlsanity.com/paper/2301.08390/) (2023)
- [Bridging Few-Shot Learning and Adaptation: New Challenges of Support-Query Shift](https://mlsanity.com/paper/2105.11804/) (2021)
- [Towards Practical Few-Shot Query Sets: Transductive Minimum Description Length Inference](https://mlsanity.com/paper/2210.14545/) (2022)
- [Trustworthy Few-Shot Transfer of Medical VLMs through Split Conformal Prediction](https://mlsanity.com/paper/2506.17503/) (2025)
- [Prototype Completion for Few-Shot Learning](https://mlsanity.com/paper/2108.05010/) (2021)
- [Fine-Grained Few Shot Learning with Foreground Object Transformation](https://mlsanity.com/paper/2109.05719/) (2021)
- [MHFC: Multi-Head Feature Collaboration for Few-Shot Learning](https://mlsanity.com/paper/2109.07785/) (2021)
- [MDFM: Multi-Decision Fusing Model for Few-Shot Learning](https://mlsanity.com/paper/2112.00690/) (2021)
- [GCT: Graph Co-Training for Semi-Supervised Few-Shot Learning](https://mlsanity.com/paper/2203.07738/) (2022)
- [Transductive Few-Shot Learning: Clustering is All You Need?](https://mlsanity.com/paper/2106.09516/) (2021)
- [TransBoost: Improving the Best ImageNet Performance using Deep Transduction](https://mlsanity.com/paper/2205.13331/) (2022)
- [Model-Agnostic Few-Shot Open-Set Recognition](https://mlsanity.com/paper/2206.09236/) (2022)
- [Meta-Learning with Heterogeneous Tasks](https://mlsanity.com/paper/2410.18894/) (2024)
- [Generative Risk Minimization for Out-of-Distribution Generalization on Graphs](https://mlsanity.com/paper/2502.07968/) (2025)
- [Mutual-Information Based Few-Shot Classification](https://mlsanity.com/paper/2106.12252/) (2021)
- [Transductive image segmentation: Self-training and effect of uncertainty estimation](https://mlsanity.com/paper/2107.08964/) (2021)

## Abstract (arXiv)

We introduce Transductive Infomation Maximization (TIM) for few-shot learning. Our method maximizes the mutual information between the query features and their label predictions for a given few-shot task, in conjunction with a supervision loss based on the support set. Furthermore, we propose a new alternating-direction solver for our mutual-information loss, which substantially speeds up transductive-inference convergence over gradient-based optimization, while yielding similar accuracy. TIM inference is modular: it can be used on top of any base-training feature extractor. Following standard transductive few-shot settings, our comprehensive experiments demonstrate that TIM outperforms state-of-the-art methods significantly across various datasets and networks, while used on top of a fixed feature extractor trained with simple cross-entropy on the base classes, without resorting to complex meta-learning schemes. It consistently brings between 2% and 5% improvement in accuracy over the best performing method, not only on all the well-established few-shot benchmarks but also on more challenging scenarios,with domain shifts and larger numbers of classes.