# An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learning

arXiv: https://arxiv.org/abs/1904.08479 · HTML: https://mlsanity.com/paper/1904.08479/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2019.

## Built on by (most cited first, 30 of 38)

- [Few-Shot Incremental Learning with Continually Evolved Classifiers](https://mlsanity.com/paper/2104.03047/) (2021)
- [Free Lunch for Few-shot Learning: Distribution Calibration](https://mlsanity.com/paper/2101.06395/) (2021)
- [Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima](https://mlsanity.com/paper/2111.01549/) (2021)
- [RMM: Reinforced Memory Management for Class-Incremental Learning](https://mlsanity.com/paper/2301.05792/) (2023)
- [Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning](https://mlsanity.com/paper/2110.03909/) (2021)
- [Learning Dynamic Alignment via Meta-filter for Few-shot Learning](https://mlsanity.com/paper/2103.13582/) (2021)
- [Generating Representative Samples for Few-Shot Classification](https://mlsanity.com/paper/2205.02918/) (2022)
- [Realistic Evaluation of Transductive Few-Shot Learning](https://mlsanity.com/paper/2204.11181/) (2022)
- [Transductive Few-Shot Classification on the Oblique Manifold](https://mlsanity.com/paper/2108.04009/) (2021)
- [Meta Navigator: Search for a Good Adaptation Policy for Few-shot Learning](https://mlsanity.com/paper/2109.05749/) (2021)
- [Few-Shot Learning with a Strong Teacher](https://mlsanity.com/paper/2107.00197/) (2021)
- [EASY: Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredients](https://mlsanity.com/paper/2201.09699/) (2022)
- [Hybrid Graph Neural Networks for Few-Shot Learning](https://mlsanity.com/paper/2112.06538/) (2021)
- [Adaptive Distribution Calibration for Few-Shot Learning with Hierarchical Optimal Transport](https://mlsanity.com/paper/2210.04144/) (2022)
- [Multimodal Prototype-Enhanced Network for Few-Shot Action Recognition](https://mlsanity.com/paper/2212.04873/) (2022)
- [Disentangled Feature Representation for Few-shot Image Classification](https://mlsanity.com/paper/2109.12548/) (2021)
- [A Comprehensive Review of Few-shot Action Recognition](https://mlsanity.com/paper/2407.14744/) (2024)
- [DMN4: Few-shot Learning via Discriminative Mutual Nearest Neighbor Neural Network](https://mlsanity.com/paper/2103.08160/) (2021)
- [Model ensemble instead of prompt fusion: a sample-specific knowledge transfer method for few-shot prompt tuning](https://mlsanity.com/paper/2210.12587/) (2022)
- [RankDNN: Learning to Rank for Few-shot Learning](https://mlsanity.com/paper/2211.15320/) (2022)
- [ESPT: A Self-Supervised Episodic Spatial Pretext Task for Improving Few-Shot Learning](https://mlsanity.com/paper/2304.13287/) (2023)
- [GDC- Generalized Distribution Calibration for Few-Shot Learning](https://mlsanity.com/paper/2204.05230/) (2022)
- [Transductive Few-Shot Learning: Clustering is All You Need?](https://mlsanity.com/paper/2106.09516/) (2021)
- [Learning Instance and Task-Aware Dynamic Kernels for Few Shot Learning](https://mlsanity.com/paper/2112.03494/) (2021)
- [Worst Case Matters for Few-Shot Recognition](https://mlsanity.com/paper/2203.06574/) (2022)
- [PatchMix Augmentation to Identify Causal Features in Few-shot Learning](https://mlsanity.com/paper/2211.16019/) (2022)
- [Few-shot Learning as Cluster-induced Voronoi Diagrams: A Geometric Approach](https://mlsanity.com/paper/2202.02471/) (2022)
- [BR-NPA: A Non-Parametric High-Resolution Attention Model to improve the Interpretability of Attention](https://mlsanity.com/paper/2106.02566/) (2021)
- [Multi-Level Contrastive Learning for Few-Shot Problems](https://mlsanity.com/paper/2107.07608/) (2021)
- [Learning Class-level Prototypes for Few-shot Learning](https://mlsanity.com/paper/2108.11072/) (2021)

## Abstract (arXiv)

Few-shot learning aims to train efficient predictive models with a few examples. The lack of training data leads to poor models that perform high-variance or low-confidence predictions. In this paper, we propose to meta-learn the ensemble of epoch-wise empirical Bayes models (E3BM) to achieve robust predictions. "Epoch-wise" means that each training epoch has a Bayes model whose parameters are specifically learned and deployed. "Empirical" means that the hyperparameters, e.g., used for learning and ensembling the epoch-wise models, are generated by hyperprior learners conditional on task-specific data. We introduce four kinds of hyperprior learners by considering inductive vs. transductive, and epoch-dependent vs. epoch-independent, in the paradigm of meta-learning. We conduct extensive experiments for five-class few-shot tasks on three challenging benchmarks: miniImageNet, tieredImageNet, and FC100, and achieve top performance using the epoch-dependent transductive hyperprior learner, which captures the richest information. Our ablation study shows that both "epoch-wise ensemble" and "empirical" encourage high efficiency and robustness in the model performance.