# Demystification of Few-shot and One-shot Learning

arXiv: https://arxiv.org/abs/2104.12174 · HTML: https://mlsanity.com/paper/2104.12174/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 20 references that are in the map.
- First posted on arXiv 25 Apr 2021.

## Builds on (most cited first, 13 of 13)

- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [The Unreasonable Effectiveness of Deep Learning in Artificial Intelligence](https://mlsanity.com/paper/2002.04806/) (2020)
- [Stochastic Configuration Networks: Fundamentals and Algorithms](https://mlsanity.com/paper/1702.03180/) (2017)
- [Correction of AI systems by linear discriminants: Probabilistic foundations](https://mlsanity.com/paper/1811.05321/) (2018)
- [Stochastic Separation Theorems](https://mlsanity.com/paper/1703.01203/) (2017)
- [The unreasonable effectiveness of small neural ensembles in high-dimensional brain](https://mlsanity.com/paper/1809.07656/) (2018)
- [General stochastic separation theorems with optimal bounds](https://mlsanity.com/paper/2010.05241/) (2020)
- [Knowledge Transfer Between Artificial Intelligence Systems](https://mlsanity.com/paper/1709.01547/) (2017)
- [High--Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality](https://mlsanity.com/paper/2001.04959/) (2020)
- [High-dimensional brain. A tool for encoding and rapid learning of memories by single neurons](https://mlsanity.com/paper/1710.11227/) (2017)

## Built on by (most cited first, 10 of 10)

- [High-dimensional separability for one- and few-shot learning](https://mlsanity.com/paper/2106.15416/) (2021)
- [Exploring and Learning Structure: Active Inference Approach in Navigational Agents](https://mlsanity.com/paper/2408.05982/) (2024)
- [Learning Dynamic Cognitive Map with Autonomous Navigation](https://mlsanity.com/paper/2411.08447/) (2024)
- [Learning from scarce information: using synthetic data to classify Roman fine ware pottery](https://mlsanity.com/paper/2107.01401/) (2021)
- [Towards a mathematical understanding of learning from few examples with nonlinear feature maps](https://mlsanity.com/paper/2211.03607/) (2022)
- [Efficient Labelling of Affective Video Datasets via Few-Shot & Multi-Task Contrastive Learning](https://mlsanity.com/paper/2308.02173/) (2023)
- [Aligning Generalisation Between Humans and Machines](https://mlsanity.com/paper/2411.15626/) (2024)
- [Quasi-orthogonality and intrinsic dimensions as measures of learning and generalisation](https://mlsanity.com/paper/2203.16687/) (2022)
- [Learning from few examples with nonlinear feature maps](https://mlsanity.com/paper/2203.16935/) (2022)
- [Beyond Rebalancing: Benchmarking Binary Classifiers Under Class Imbalance Without Rebalancing Techniques](https://mlsanity.com/paper/2509.07605/) (2025)

## Abstract (arXiv)

Few-shot and one-shot learning have been the subject of active and intensive research in recent years, with mounting evidence pointing to successful implementation and exploitation of few-shot learning algorithms in practice. Classical statistical learning theories do not fully explain why few- or one-shot learning is at all possible since traditional generalisation bounds normally require large training and testing samples to be meaningful. This sharply contrasts with numerous examples of successful one- and few-shot learning systems and applications. In this work we present mathematical foundations for a theory of one-shot and few-shot learning and reveal conditions specifying when such learning schemes are likely to succeed. Our theory is based on intrinsic properties of high-dimensional spaces. We show that if the ambient or latent decision space of a learning machine is sufficiently high-dimensional than a large class of objects in this space can indeed be easily learned from few examples provided that certain data non-concentration conditions are met.