# Relative intrinsic dimensionality is intrinsic to learning

arXiv: https://arxiv.org/abs/2311.07579 · HTML: https://mlsanity.com/paper/2311.07579/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 13 references that are in the map.
- First posted on arXiv 10 Oct 2023.

## Builds on (most cited first, 7 of 7)

- [Scikit-dimension: a Python package for intrinsic dimension estimation](https://mlsanity.com/paper/2109.02596/) (2021)
- [Blessing of dimensionality: mathematical foundations of the statistical physics of data](https://mlsanity.com/paper/1801.03421/) (2018)
- [Estimating the effective dimension of large biological datasets using Fisher separability analysis](https://mlsanity.com/paper/1901.06328/) (2019)
- [The More, the Merrier: the Blessing of Dimensionality for Learning Large Gaussian Mixtures](https://mlsanity.com/paper/1311.2891/) (2013)
- [Correction of AI systems by linear discriminants: Probabilistic foundations](https://mlsanity.com/paper/1811.05321/) (2018)
- [Stochastic Separation Theorems](https://mlsanity.com/paper/1703.01203/) (2017)
- [Towards a mathematical understanding of learning from few examples with nonlinear feature maps](https://mlsanity.com/paper/2211.03607/) (2022)

## Built on by (most cited first, 6 of 6)

- [Stealth edits to large language models](https://mlsanity.com/paper/2406.12670/) (2024)
- [What is the textitintrinsic dimension of your binary data? -- and how to compute it quickly](https://mlsanity.com/paper/2404.06326/) (2024)
- [Networks with Finite VC Dimension: Pro and Contra](https://mlsanity.com/paper/2502.02679/) (2025)
- [When fractional quasi p-norms concentrate](https://mlsanity.com/paper/2505.19635/) (2025)
- [Theoretical Lower Bounds on the Robustness of Deep ReLU Networks](https://mlsanity.com/paper/2602.18674/) (2026)
- [Low-power analogue neural networks with trainable nonlinear connections for continuous control](https://mlsanity.com/paper/2606.23742/) (2026)

## Abstract (arXiv)

High dimensional data can have a surprising property: pairs of data points may be easily separated from each other, or even from arbitrary subsets, with high probability using just simple linear classifiers. However, this is more of a rule of thumb than a reliable property as high dimensionality alone is neither necessary nor sufficient for successful learning. Here, we introduce a new notion of the intrinsic dimension of a data distribution, which precisely captures the separability properties of the data. For this intrinsic dimension, the rule of thumb above becomes a law: high intrinsic dimension guarantees highly separable data. We extend this notion to that of the relative intrinsic dimension of two data distributions, which we show provides both upper and lower bounds on the probability of successfully learning and generalising in a binary classification problem