# High-dimensional separability for one- and few-shot learning

arXiv: https://arxiv.org/abs/2106.15416 · HTML: https://mlsanity.com/paper/2106.15416/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 127 references that are in the map.
- First posted on arXiv 28 Jun 2021.

## Builds on (most cited first, 25 of 25)

- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019)
- [Blessing of dimensionality: mathematical foundations of the statistical physics of data](https://mlsanity.com/paper/1801.03421/) (2018)
- [Self-Correcting Models for Model-Based Reinforcement Learning](https://mlsanity.com/paper/1612.06018/) (2016)
- [A Compact Embedding for Facial Expression Similarity](https://mlsanity.com/paper/1811.11283/) (2018)
- [The More, the Merrier: the Blessing of Dimensionality for Learning Large Gaussian Mixtures](https://mlsanity.com/paper/1311.2891/) (2013)
- [Correction of AI systems by linear discriminants: Probabilistic foundations](https://mlsanity.com/paper/1811.05321/) (2018)
- [On Adversarial Examples and Stealth Attacks in Artificial Intelligence Systems](https://mlsanity.com/paper/2004.04479/) (2020)
- [Fractional norms and quasinorms do not help to overcome the curse of dimensionality](https://mlsanity.com/paper/2004.14230/) (2020)
- [The Feasibility and Inevitability of Stealth Attacks](https://mlsanity.com/paper/2106.13997/) (2021)
- [Stochastic Separation Theorems](https://mlsanity.com/paper/1703.01203/) (2017)
- [The unreasonable effectiveness of small neural ensembles in high-dimensional brain](https://mlsanity.com/paper/1809.07656/) (2018)
- [Demystification of Few-shot and One-shot Learning](https://mlsanity.com/paper/2104.12174/) (2021)
- [General stochastic separation theorems with optimal bounds](https://mlsanity.com/paper/2010.05241/) (2020)
- [Knowledge Transfer Between Artificial Intelligence Systems](https://mlsanity.com/paper/1709.01547/) (2017)
- [High--Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality](https://mlsanity.com/paper/2001.04959/) (2020)
- [AI Failures: A Review of Underlying Issues](https://mlsanity.com/paper/2008.04073/) (2020)
- [Robust And Scalable Learning Of Complex Dataset Topologies Via Elpigraph](https://mlsanity.com/paper/1804.07580/) (2018)
- [How deep should be the depth of convolutional neural networks: a backyard dog case study](https://mlsanity.com/paper/1805.01516/) (2018)
- [Blessing of dimensionality at the edge](https://mlsanity.com/paper/1910.00445/) (2019)
- [Augmented Artificial Intelligence: a Conceptual Framework](https://mlsanity.com/paper/1802.02172/) (2018)

## Built on by (most cited first, 11 of 11)

- [Scikit-dimension: a Python package for intrinsic dimension estimation](https://mlsanity.com/paper/2109.02596/) (2021)
- [The Boundaries of Verifiable Accuracy, Robustness, and Generalisation in Deep Learning](https://mlsanity.com/paper/2309.07072/) (2023)
- [Learning from scarce information: using synthetic data to classify Roman fine ware pottery](https://mlsanity.com/paper/2107.01401/) (2021)
- [Towards a mathematical understanding of learning from few examples with nonlinear feature maps](https://mlsanity.com/paper/2211.03607/) (2022)
- [Weakly Supervised Learners for Correction of AI Errors with Provable Performance Guarantees](https://mlsanity.com/paper/2402.00899/) (2024)
- [Learning from few examples with nonlinear feature maps](https://mlsanity.com/paper/2203.16935/) (2022)
- [Domain Adaptation Principal Component Analysis: base linear method for learning with out-of-distribution data](https://mlsanity.com/paper/2208.13290/) (2022)
- [XXAI: Towards eXplicitly eXplainable Artificial Intelligence](https://mlsanity.com/paper/2401.03093/) (2024)
- [When fractional quasi p-norms concentrate](https://mlsanity.com/paper/2505.19635/) (2025)
- [Error correction in multiclass image classification of facial emotion on unbalanced samples](https://mlsanity.com/paper/2510.03337/) (2025)
- [Stochastic Separability of Embedding Manifolds](https://mlsanity.com/paper/2608.22874/) (2026)

## Abstract (arXiv)

This work is driven by a practical question: corrections of Artificial Intelligence (AI) errors. These corrections should be quick and non-iterative. To solve this problem without modification of a legacy AI system, we propose special `external' devices, correctors. Elementary correctors consist of two parts, a classifier that separates the situations with high risk of error from the situations in which the legacy AI system works well and a new decision for situations with potential errors. Input signals for the correctors can be the inputs of the legacy AI system, its internal signals, and outputs. If the intrinsic dimensionality of data is high enough then the classifiers for correction of small number of errors can be very simple. According to the blessing of dimensionality effects, even simple and robust Fisher's discriminants can be used for one-shot learning of AI correctors. Stochastic separation theorems provide the mathematical basis for this one-short learning. However, as the number of correctors needed grows, the cluster structure of data becomes important and a new family of stochastic separation theorems is required. We refuse the classical hypothesis of the regularity of the data distribution and assume that the data can have a fine-grained structure with many clusters and peaks in the probability density. New stochastic separation theorems for data with fine-grained structure are formulated and proved. The multi-correctors for granular data are proposed. The advantages of the multi-corrector technology were demonstrated by examples of correcting errors and learning new classes of objects by a deep convolutional neural network on the CIFAR-10 dataset. The key problems of the non-classical high-dimensional data analysis are reviewed together with the basic preprocessing steps including supervised, semi-supervised and domain adaptation Principal Component Analysis.