# Diversity with Cooperation: Ensemble Methods for Few-Shot Classification

arXiv: https://arxiv.org/abs/1903.11341 · HTML: https://mlsanity.com/paper/1903.11341/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2019.

## Built on by (most cited first, 30 of 70)

- [Deep Model Fusion: A Survey](https://mlsanity.com/paper/2309.15698/) (2023)
- [Nested Collaborative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2203.15359/) (2022)
- [Pareto Self-Supervised Training for Few-Shot Learning](https://mlsanity.com/paper/2104.07841/) (2021)
- [Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition](https://mlsanity.com/paper/2101.02833/) (2021)
- [DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation](https://mlsanity.com/paper/2101.05544/) (2021)
- [Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillation](https://mlsanity.com/paper/2209.04599/) (2022)
- [Learning Dynamic Alignment via Meta-filter for Few-shot Learning](https://mlsanity.com/paper/2103.13582/) (2021)
- [Generating Representative Samples for Few-Shot Classification](https://mlsanity.com/paper/2205.02918/) (2022)
- [Partner-Assisted Learning for Few-Shot Image Classification](https://mlsanity.com/paper/2109.07607/) (2021)
- [Few-shot Image Classification: Just Use a Library of Pre-trained Feature Extractors and a Simple Classifier](https://mlsanity.com/paper/2101.00562/) (2021)
- [Memorizing Complementation Network for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2208.05610/) (2022)
- [Class-Balanced Distillation for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2104.05279/) (2021)
- [MMG-Ego4D: Multi-Modal Generalization in Egocentric Action Recognition](https://mlsanity.com/paper/2305.07214/) (2023)
- [Matching Feature Sets for Few-Shot Image Classification](https://mlsanity.com/paper/2204.00949/) (2022)
- [A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applications](https://mlsanity.com/paper/2111.01968/) (2021)
- [Transductive Few-Shot Classification on the Oblique Manifold](https://mlsanity.com/paper/2108.04009/) (2021)
- [Selective Ensembles for Consistent Predictions](https://mlsanity.com/paper/2111.08230/) (2021)
- [Integrating Object-aware and Interaction-aware Knowledge for Weakly Supervised Scene Graph Generation](https://mlsanity.com/paper/2208.01834/) (2022)
- [Meta Navigator: Search for a Good Adaptation Policy for Few-shot Learning](https://mlsanity.com/paper/2109.05749/) (2021)
- [Self-Supervision Can Be a Good Few-Shot Learner](https://mlsanity.com/paper/2207.09176/) (2022)
- [Student Surpasses Teacher: Imitation Attack for Black-Box NLP APIs](https://mlsanity.com/paper/2108.13873/) (2021)
- [Semantically Meaningful Class Prototype Learning for One-Shot Image Semantic Segmentation](https://mlsanity.com/paper/2102.10935/) (2021)
- [On the Importance of Distractors for Few-Shot Classification](https://mlsanity.com/paper/2109.09883/) (2021)
- [NCL++: Nested Collaborative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2306.16709/) (2023)
- [Diagnosing Ensemble Few-Shot Classifiers](https://mlsanity.com/paper/2206.04372/) (2022)
- [Feature Space Particle Inference for Neural Network Ensembles](https://mlsanity.com/paper/2206.00944/) (2022)
- [Model ensemble instead of prompt fusion: a sample-specific knowledge transfer method for few-shot prompt tuning](https://mlsanity.com/paper/2210.12587/) (2022)
- [GeNIe: Generative Hard Negative Images Through Diffusion](https://mlsanity.com/paper/2312.02548/) (2023)
- [On the Importance of Firth Bias Reduction in Few-Shot Classification](https://mlsanity.com/paper/2110.02529/) (2021)
- [MHFC: Multi-Head Feature Collaboration for Few-Shot Learning](https://mlsanity.com/paper/2109.07785/) (2021)

## Abstract (arXiv)

Few-shot classification consists of learning a predictive model that is able to effectively adapt to a new class, given only a few annotated samples. To solve this challenging problem, meta-learning has become a popular paradigm that advocates the ability to "learn to adapt". Recent works have shown, however, that simple learning strategies without meta-learning could be competitive. In this paper, we go a step further and show that by addressing the fundamental high-variance issue of few-shot learning classifiers, it is possible to significantly outperform current meta-learning techniques. Our approach consists of designing an ensemble of deep networks to leverage the variance of the classifiers, and introducing new strategies to encourage the networks to cooperate, while encouraging prediction diversity. Evaluation is conducted on the mini-ImageNet and CUB datasets, where we show that even a single network obtained by distillation yields state-of-the-art results.