# Blessing of dimensionality: mathematical foundations of the statistical physics of data

arXiv: https://arxiv.org/abs/1801.03421 · HTML: https://mlsanity.com/paper/1801.03421/

## Key facts

- Cited by 55 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 10 Jan 2018.

## Built on by (most cited first, 30 of 55)

- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations](https://mlsanity.com/paper/2111.06077/) (2021)
- [Independent mechanism analysis, a new concept?](https://mlsanity.com/paper/2106.05200/) (2021)
- [Scikit-dimension: a Python package for intrinsic dimension estimation](https://mlsanity.com/paper/2109.02596/) (2021)
- [Computing on Functions Using Randomized Vector Representations](https://mlsanity.com/paper/2109.03429/) (2021)
- [Vector Symbolic Architectures as a Computing Framework for Emerging Hardware](https://mlsanity.com/paper/2106.05268/) (2021)
- [High-dimensional separability for one- and few-shot learning](https://mlsanity.com/paper/2106.15416/) (2021)
- [Deconfounded Score Method: Scoring DAGs with Dense Unobserved Confounding](https://mlsanity.com/paper/2103.15106/) (2021)
- [Learning and Meshing from Deep Implicit Surface Networks Using an Efficient Implementation of Analytic Marching](https://mlsanity.com/paper/2106.10031/) (2021)
- [Hyperseed: Unsupervised Learning with Vector Symbolic Architectures](https://mlsanity.com/paper/2110.08343/) (2021)
- [Relative intrinsic dimensionality is intrinsic to learning](https://mlsanity.com/paper/2311.07579/) (2023)
- [A Metamodel and Framework for Artificial General Intelligence From Theory to Practice](https://mlsanity.com/paper/2102.06112/) (2021)
- [Neurosymbolic Systems of Perception & Cognition: The Role of Attention](https://mlsanity.com/paper/2112.01603/) (2021)
- [Bayesian Active Learning for Discrete Latent Variable Models](https://mlsanity.com/paper/2202.13426/) (2022)
- [Towards Characterizing Domain Counterfactuals For Invertible Latent Causal Models](https://mlsanity.com/paper/2306.11281/) (2023)
- [Position: Cracking the Code of Cascading Disparity Towards Marginalized Communities](https://mlsanity.com/paper/2406.01757/) (2024)
- [Unsupervised Dialogue Topic Segmentation in Hyperdimensional Space](https://mlsanity.com/paper/2308.10464/) (2023)
- [Towards a mathematical understanding of learning from few examples with nonlinear feature maps](https://mlsanity.com/paper/2211.03607/) (2022)
- [Defining Neural Network Architecture through Polytope Structures of Dataset](https://mlsanity.com/paper/2402.02407/) (2024)
- [Hyperdimensional computing: a fast, robust and interpretable paradigm for biological data](https://mlsanity.com/paper/2402.17572/) (2024)
- [Adaptive k-nearest neighbor classifier based on the local estimation of the shape operator](https://mlsanity.com/paper/2409.05084/) (2024)
- [Robust training of implicit generative models for multivariate and heavy-tailed distributions with an invariant statistical loss](https://mlsanity.com/paper/2410.22381/) (2024)
- [HiPoNet: A Multi-View Simplicial Complex Network for High Dimensional Point-Cloud and Single-Cell Data](https://mlsanity.com/paper/2502.07746/) (2025)
- [Rollout-LaSDI: Enhancing the long-term accuracy of Latent Space Dynamics](https://mlsanity.com/paper/2509.08191/) (2025)
- [Review of Hallucination Understanding in Large Language and Vision Models](https://mlsanity.com/paper/2510.00034/) (2025)
- [Higher-Order LaSDI: Reduced Order Modeling with Multiple Time Derivatives](https://mlsanity.com/paper/2512.15997/) (2025)
- [Quasi-orthogonality and intrinsic dimensions as measures of learning and generalisation](https://mlsanity.com/paper/2203.16687/) (2022)
- [A Modular and Adaptive System for Business Email Compromise Detection](https://mlsanity.com/paper/2308.10776/) (2023)
- [Unbiasing Enhanced Sampling on a High-dimensional Free Energy Surface with Deep Generative Model](https://mlsanity.com/paper/2312.09404/) (2023)
- [Weakly Supervised Learners for Correction of AI Errors with Provable Performance Guarantees](https://mlsanity.com/paper/2402.00899/) (2024)
- [A Mathematical Framework for the Problem of Security for Cognition in Neurotechnology](https://mlsanity.com/paper/2403.07945/) (2024)

## Abstract (arXiv)

The concentration of measure phenomena were discovered as the mathematical background of statistical mechanics at the end of the XIX - beginning of the XX century and were then explored in mathematics of the XX-XXI centuries. At the beginning of the XXI century, it became clear that the proper utilisation of these phenomena in machine learning might transform the curse of dimensionality into the blessing of dimensionality. This paper summarises recently discovered phenomena of measure concentration which drastically simplify some machine learning problems in high dimension, and allow us to correct legacy artificial intelligence systems. The classical concentration of measure theorems state that i.i.d. random points are concentrated in a thin layer near a surface (a sphere or equators of a sphere, an average or median level set of energy or another Lipschitz function, etc.). The new stochastic separation theorems describe the thin structure of these thin layers: the random points are not only concentrated in a thin layer but are all linearly separable from the rest of the set, even for exponentially large random sets. The linear functionals for separation of points can be selected in the form of the linear Fisher's discriminant. All artificial intelligence systems make errors. Non-destructive correction requires separation of the situations (samples) with errors from the samples corresponding to correct behaviour by a simple and robust classifier. The stochastic separation theorems provide us by such classifiers and a non-iterative (one-shot) procedure for learning.