# Empirical Bayes Transductive Meta-Learning with Synthetic Gradients

arXiv: https://arxiv.org/abs/2004.12696 · HTML: https://mlsanity.com/paper/2004.12696/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Apr 2020.

## Built on by (most cited first, 30 of 48)

- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference](https://mlsanity.com/paper/2204.07305/) (2022)
- [Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning](https://mlsanity.com/paper/2110.03909/) (2021)
- [Transductive Few-shot Learning with Prototype-based Label Propagation by Iterative Graph Refinement](https://mlsanity.com/paper/2304.11598/) (2023)
- [Realistic Evaluation of Transductive Few-Shot Learning](https://mlsanity.com/paper/2204.11181/) (2022)
- [Transductive Few-Shot Classification on the Oblique Manifold](https://mlsanity.com/paper/2108.04009/) (2021)
- [Context-Aware Meta-Learning](https://mlsanity.com/paper/2310.10971/) (2023)
- [EASY: Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredients](https://mlsanity.com/paper/2201.09699/) (2022)
- [POODLE: Improving Few-shot Learning via Penalizing Out-of-Distribution Samples](https://mlsanity.com/paper/2206.04679/) (2022)
- [Few-Shot and Continual Learning with Attentive Independent Mechanisms](https://mlsanity.com/paper/2107.14053/) (2021)
- [MetaNODE: Prototype Optimization as a Neural ODE for Few-Shot Learning](https://mlsanity.com/paper/2103.14341/) (2021)
- [SURE: SUrvey REcipes for building reliable and robust deep networks](https://mlsanity.com/paper/2403.00543/) (2024)
- [Unsupervised Embedding Adaptation via Early-Stage Feature Reconstruction for Few-Shot Classification](https://mlsanity.com/paper/2106.11486/) (2021)
- [Prototype Completion for Few-Shot Learning](https://mlsanity.com/paper/2108.05010/) (2021)
- [Transductive Decoupled Variational Inference for Few-Shot Classification](https://mlsanity.com/paper/2208.10559/) (2022)
- [Discriminative Sample-Guided and Parameter-Efficient Feature Space Adaptation for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2403.04492/) (2024)
- [MHFC: Multi-Head Feature Collaboration for Few-Shot Learning](https://mlsanity.com/paper/2109.07785/) (2021)
- [Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptile](https://mlsanity.com/paper/2206.01944/) (2022)
- [ESPT: A Self-Supervised Episodic Spatial Pretext Task for Improving Few-Shot Learning](https://mlsanity.com/paper/2304.13287/) (2023)
- [BECLR: Batch Enhanced Contrastive Few-Shot Learning](https://mlsanity.com/paper/2402.02444/) (2024)
- [Benchmarking Spurious Bias in Few-Shot Image Classifiers](https://mlsanity.com/paper/2409.02882/) (2024)
- [MDFM: Multi-Decision Fusing Model for Few-Shot Learning](https://mlsanity.com/paper/2112.00690/) (2021)
- [MetaCoCo: A New Few-Shot Classification Benchmark with Spurious Correlation](https://mlsanity.com/paper/2404.19644/) (2024)
- [EBPO: Empirical Bayes Shrinkage for Stabilizing Group-Relative Policy Optimization](https://mlsanity.com/paper/2602.05165/) (2026)
- [Adaptive Pseudo-Label Refinement by Negative Ensemble Learning for Source-Free Unsupervised Domain Adaptation](https://mlsanity.com/paper/2103.15973/) (2021)
- [Transductive Few-Shot Learning: Clustering is All You Need?](https://mlsanity.com/paper/2106.09516/) (2021)
- [DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learning](https://mlsanity.com/paper/2305.07892/) (2023)
- [Quantum-Aided Meta-Learning for Bayesian Binary Neural Networks via Born Machines](https://mlsanity.com/paper/2203.17089/) (2022)
- [Enhancing Prototypical Few-Shot Learning by Leveraging the Local-Level Strategy](https://mlsanity.com/paper/2111.04331/) (2021)
- [Meta-free few-shot learning via representation learning with weight averaging](https://mlsanity.com/paper/2204.12466/) (2022)

## Abstract (arXiv)

We propose a meta-learning approach that learns from multiple tasks in a transductive setting, by leveraging the unlabeled query set in addition to the support set to generate a more powerful model for each task. To develop our framework, we revisit the empirical Bayes formulation for multi-task learning. The evidence lower bound of the marginal log-likelihood of empirical Bayes decomposes as a sum of local KL divergences between the variational posterior and the true posterior on the query set of each task. We derive a novel amortized variational inference that couples all the variational posteriors via a meta-model, which consists of a synthetic gradient network and an initialization network. Each variational posterior is derived from synthetic gradient descent to approximate the true posterior on the query set, although where we do not have access to the true gradient. Our results on the Mini-ImageNet and CIFAR-FS benchmarks for episodic few-shot classification outperform previous state-of-the-art methods. Besides, we conduct two zero-shot learning experiments to further explore the potential of the synthetic gradient.