# Estimating Dimensionality of Neural Representations from Finite Samples

arXiv: https://arxiv.org/abs/2509.26560 · HTML: https://mlsanity.com/paper/2509.26560/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 46 references that are in the map.
- First posted on arXiv 30 Sep 2025.

## Builds on (most cited first, 16 of 16)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Layer by Layer: Uncovering Hidden Representations in Language Models](https://mlsanity.com/paper/2502.02013/) (2025)
- [Estimating the intrinsic dimension of datasets by a minimal neighborhood information](https://mlsanity.com/paper/1803.06992/) (2018)
- [Are Sparse Autoencoders Useful? A Case Study in Sparse Probing](https://mlsanity.com/paper/2502.16681/) (2025)
- [The geometry of hidden representations of large transformer models](https://mlsanity.com/paper/2302.00294/) (2023)
- [Classification and Geometry of General Perceptual Manifolds](https://mlsanity.com/paper/1710.06487/) (2017)
- [Learning Efficient Coding of Natural Images with Maximum Manifold Capacity Representations](https://mlsanity.com/paper/2303.03307/) (2023)
- [What Representational Similarity Measures Imply about Decodable Information](https://mlsanity.com/paper/2411.08197/) (2024)
- [The Geometry of Prompting: Unveiling Distinct Mechanisms of Task Adaptation in Language Models](https://mlsanity.com/paper/2502.08009/) (2025)
- [NeuroAI for AI Safety](https://mlsanity.com/paper/2411.18526/) (2024)
- [The Geometry of Harmfulness in LLMs through Subconcept Probing](https://mlsanity.com/paper/2507.21141/) (2025)
- [What does a deep neural network confidently perceive? The effective dimension of high certainty class manifolds and their low confidence boundaries](https://mlsanity.com/paper/2210.05546/) (2022)
- [Stimuli reduce the dimensionality of cortical activity](https://mlsanity.com/paper/1509.03621/) (2015)
- [Quality of internal representation shapes learning performance in feedback neural networks](https://mlsanity.com/paper/2011.06066/) (2020)
- [Probing Biological and Artificial Neural Networks with Task-dependent Neural Manifolds](https://mlsanity.com/paper/2312.14285/) (2023)
- [Estimating the Spectral Moments of the Kernel Integral Operator from Finite Sample Matrices](https://mlsanity.com/paper/2410.17998/) (2024)

## Abstract (arXiv)

The global dimensionality of a neural representation manifold provides rich insight into the computational process underlying both artificial and biological neural networks. However, all existing measures of global dimensionality are sensitive to the number of samples, i.e., the number of rows and columns of the sample matrix. We show that, in particular, the participation ratio of eigenvalues, a popular measure of global dimensionality, is highly biased with small sample sizes, and propose a bias-corrected estimator that is more accurate with finite samples and with noise. On synthetic data examples, we demonstrate that our estimator can recover the true known dimensionality. We apply our estimator to neural brain recordings, including calcium imaging, electrophysiological recordings, and fMRI data, and to the neural activations in a large language model and show our estimator is invariant to the sample size. Finally, our estimators can additionally be used to measure the local dimensionalities of curved neural manifolds by weighting the finite samples appropriately.