# Few-shot Classification via Ensemble Learning with Multi-Order Statistics

arXiv: https://arxiv.org/abs/2305.00454 · HTML: https://mlsanity.com/paper/2305.00454/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 55 references that are in the map.
- First posted on arXiv 30 Apr 2023.

## Builds on (most cited first, 30 of 33)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?](https://mlsanity.com/paper/2003.11539/) (2020)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [Neural Additive Models: Interpretable Machine Learning with Neural Nets](https://mlsanity.com/paper/2004.13912/) (2020)
- [SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning](https://mlsanity.com/paper/1911.04623/) (2019)
- [Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation](https://mlsanity.com/paper/2001.08735/) (2020)
- [Free Lunch for Few-shot Learning: Distribution Calibration](https://mlsanity.com/paper/2101.06395/) (2021)
- [Boosting Few-Shot Visual Learning with Self-Supervision](https://mlsanity.com/paper/1906.05186/) (2019)
- [Charting the Right Manifold: Manifold Mixup for Few-shot Learning](https://mlsanity.com/paper/1907.12087/) (2019)
- [Negative Margin Matters: Understanding Margin in Few-shot Classification](https://mlsanity.com/paper/2003.12060/) (2020)
- [Few-Shot Classification with Feature Map Reconstruction Networks](https://mlsanity.com/paper/2012.01506/) (2020)
- [Relational Embedding for Few-Shot Classification](https://mlsanity.com/paper/2108.09666/) (2021)
- [Diversity with Cooperation: Ensemble Methods for Few-Shot Classification](https://mlsanity.com/paper/1903.11341/) (2019)
- [Associative Alignment for Few-shot Image Classification](https://mlsanity.com/paper/1912.05094/) (2019)
- [Exploring Complementary Strengths of Invariant and Equivariant Representations for Few-Shot Learning](https://mlsanity.com/paper/2103.01315/) (2021)
- [Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classification](https://mlsanity.com/paper/2204.04567/) (2022)
- [Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning](https://mlsanity.com/paper/2110.03909/) (2021)
- [Boosting Randomized Smoothing with Variance Reduced Classifiers](https://mlsanity.com/paper/2106.06946/) (2021)
- [von Mises-Fisher Loss: An Exploration of Embedding Geometries for Supervised Learning](https://mlsanity.com/paper/2103.15718/) (2021)
- [Self-supervised Knowledge Distillation for Few-shot Learning](https://mlsanity.com/paper/2006.09785/) (2020)
- [Learning Dynamic Alignment via Meta-filter for Few-shot Learning](https://mlsanity.com/paper/2103.13582/) (2021)
- [Partner-Assisted Learning for Few-Shot Image Classification](https://mlsanity.com/paper/2109.07607/) (2021)
- [Binocular Mutual Learning for Improving Few-shot Classification](https://mlsanity.com/paper/2108.12104/) (2021)
- [Matching Feature Sets for Few-Shot Image Classification](https://mlsanity.com/paper/2204.00949/) (2022)
- [Cooperative Bi-path Metric for Few-shot Learning](https://mlsanity.com/paper/2008.04031/) (2020)
- [Adaptive Poincar'e Point to Set Distance for Few-Shot Classification](https://mlsanity.com/paper/2112.01719/) (2021)
- [Hybrid Graph Neural Networks for Few-Shot Learning](https://mlsanity.com/paper/2112.06538/) (2021)

## Built on by (most cited first, 3 of 3)

- [MIKE: A New Benchmark for Fine-grained Multimodal Entity Knowledge Editing](https://mlsanity.com/paper/2402.14835/) (2024)
- [Object-conditioned Bag of Instances for Few-Shot Personalized Instance Recognition](https://mlsanity.com/paper/2404.01397/) (2024)
- [HGEN: Heterogeneous Graph Ensemble Networks](https://mlsanity.com/paper/2509.09843/) (2025)

## Abstract (arXiv)

Transfer learning has been widely adopted for few-shot classification. Recent studies reveal that obtaining good generalization representation of images on novel classes is the key to improving the few-shot classification accuracy. To address this need, we prove theoretically that leveraging ensemble learning on the base classes can correspondingly reduce the true error in the novel classes. Following this principle, a novel method named Ensemble Learning with Multi-Order Statistics (ELMOS) is proposed in this paper. In this method, after the backbone network, we use multiple branches to create the individual learners in the ensemble learning, with the goal to reduce the storage cost. We then introduce different order statistics pooling in each branch to increase the diversity of the individual learners. The learners are optimized with supervised losses during the pre-training phase. After pre-training, features from different branches are concatenated for classifier evaluation. Extensive experiments demonstrate that each branch can complement the others and our method can produce a state-of-the-art performance on multiple few-shot classification benchmark datasets.