# Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition

arXiv: https://arxiv.org/abs/2101.02833 · HTML: https://mlsanity.com/paper/2101.02833/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 70 references that are in the map.
- First posted on arXiv 8 Jan 2021.

## Builds on (most cited first, 30 of 50)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [Meta-Learning with Latent Embedding Optimization](https://mlsanity.com/paper/1807.05960/) (2018)
- [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [Measuring Calibration in Deep Learning](https://mlsanity.com/paper/1904.01685/) (2019)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [TADAM: Task dependent adaptive metric for improved few-shot learning](https://mlsanity.com/paper/1805.10123/) (2018)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [Dynamic Few-Shot Visual Learning without Forgetting](https://mlsanity.com/paper/1804.09458/) (2018)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [Few-Shot Learning with Graph Neural Networks](https://mlsanity.com/paper/1711.04043/) (2017)
- [Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples](https://mlsanity.com/paper/1903.03096/) (2019)
- [Meta-Transfer Learning for Few-Shot Learning](https://mlsanity.com/paper/1812.02391/) (2018)
- [Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2004.10956/) (2020)
- [SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning](https://mlsanity.com/paper/1911.04623/) (2019)
- [Bayesian Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.03836/) (2018)
- [Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation](https://mlsanity.com/paper/2001.08735/) (2020)
- [Recasting Gradient-Based Meta-Learning as Hierarchical Bayes](https://mlsanity.com/paper/1801.08930/) (2018)

## Built on by (most cited first, 30 of 48)

- [Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference](https://mlsanity.com/paper/2204.07305/) (2022)
- [Variational Bayesian Last Layers](https://mlsanity.com/paper/2404.11599/) (2024)
- [MetaDiff: Meta-Learning with Conditional Diffusion for Few-Shot Learning](https://mlsanity.com/paper/2307.16424/) (2023)
- [Diffusion-Based Neural Network Weights Generation](https://mlsanity.com/paper/2402.18153/) (2024)
- [Adaptive Transformers for Robust Few-shot Cross-domain Face Anti-spoofing](https://mlsanity.com/paper/2203.12175/) (2022)
- [Rethinking Generalization in Few-Shot Classification](https://mlsanity.com/paper/2206.07267/) (2022)
- [SVL-Adapter: Self-Supervised Adapter for Vision-Language Pretrained Models](https://mlsanity.com/paper/2210.03794/) (2022)
- [Attribute Surrogates Learning and Spectral Tokens Pooling in Transformers for Few-shot Learning](https://mlsanity.com/paper/2203.09064/) (2022)
- [Wave-SAN: Wavelet based Style Augmentation Network for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2203.07656/) (2022)
- [Inductive and Transductive Few-Shot Video Classification via Appearance and Temporal Alignments](https://mlsanity.com/paper/2207.10785/) (2022)
- [Supervised Masked Knowledge Distillation for Few-Shot Transformers](https://mlsanity.com/paper/2303.15466/) (2023)
- [Context-Aware Meta-Learning](https://mlsanity.com/paper/2310.10971/) (2023)
- [TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edge](https://mlsanity.com/paper/2307.09988/) (2023)
- [A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trends](https://mlsanity.com/paper/2402.15490/) (2024)
- [Towards Task Sampler Learning for Meta-Learning](https://mlsanity.com/paper/2307.08924/) (2023)
- [On the Importance of Spatial Relations for Few-shot Action Recognition](https://mlsanity.com/paper/2308.07119/) (2023)
- [Lessons learned from the NeurIPS 2021 MetaDL challenge: Backbone fine-tuning without episodic meta-learning dominates for few-shot learning image classification](https://mlsanity.com/paper/2206.08138/) (2022)
- [On the Importance of Firth Bias Reduction in Few-Shot Classification](https://mlsanity.com/paper/2110.02529/) (2021)
- [Transductive Decoupled Variational Inference for Few-Shot Classification](https://mlsanity.com/paper/2208.10559/) (2022)
- [BECLR: Batch Enhanced Contrastive Few-Shot Learning](https://mlsanity.com/paper/2402.02444/) (2024)
- [Self-Attention Message Passing for Contrastive Few-Shot Learning](https://mlsanity.com/paper/2210.06339/) (2022)
- [Enhancing Few-shot Image Classification with Cosine Transformer](https://mlsanity.com/paper/2211.06828/) (2022)
- [Meta-DM: Applications of Diffusion Models on Few-Shot Learning](https://mlsanity.com/paper/2305.08092/) (2023)
- [CLCE: An Approach to Refining Cross-Entropy and Contrastive Learning for Optimized Learning Fusion](https://mlsanity.com/paper/2402.14551/) (2024)
- [Gaussian Process Meta Few-shot Classifier Learning via Linear Discriminant Laplace Approximation](https://mlsanity.com/paper/2111.05392/) (2021)
- [Unleash Model Potential: Bootstrapped Meta Self-supervised Learning](https://mlsanity.com/paper/2308.14267/) (2023)
- [How Sensitive are Meta-Learners to Dataset Imbalance?](https://mlsanity.com/paper/2104.05344/) (2021)
- [Bayesian Evidential Learning for Few-Shot Classification](https://mlsanity.com/paper/2207.13137/) (2022)
- [BaseTransformers: Attention over base data-points for One Shot Learning](https://mlsanity.com/paper/2210.02476/) (2022)
- [Exploring Active Learning in Meta-Learning: Enhancing Context Set Labeling](https://mlsanity.com/paper/2311.02879/) (2023)

## Abstract (arXiv)

Current state-of-the-art few-shot learners focus on developing effective training procedures for feature representations, before using simple, e.g. nearest centroid, classifiers. In this paper, we take an orthogonal approach that is agnostic to the features used and focus exclusively on meta-learning the actual classifier layer. Specifically, we introduce MetaQDA, a Bayesian meta-learning generalization of the classic quadratic discriminant analysis. This setup has several benefits of interest to practitioners: meta-learning is fast and memory-efficient, without the need to fine-tune features. It is agnostic to the off-the-shelf features chosen and thus will continue to benefit from advances in feature representations. Empirically, it leads to robust performance in cross-domain few-shot learning and, crucially for real-world applications, it leads to better uncertainty calibration in predictions.