# A Novel Approach for Intrinsic Dimension Estimation

arXiv: https://arxiv.org/abs/2503.09485 · HTML: https://mlsanity.com/paper/2503.09485/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 46 references that are in the map.
- First posted on arXiv 12 Mar 2025.

## Builds on (most cited first, 6 of 6)

- [Estimating the intrinsic dimension of datasets by a minimal neighborhood information](https://mlsanity.com/paper/1803.06992/) (2018)
- [Scikit-dimension: a Python package for intrinsic dimension estimation](https://mlsanity.com/paper/2109.02596/) (2021)
- [Intrinsic Dimensionality Estimation within Tight Localities: A Theoretical and Experimental Analysis](https://mlsanity.com/paper/2209.14475/) (2022)
- [Estimating the effective dimension of large biological datasets using Fisher separability analysis](https://mlsanity.com/paper/1901.06328/) (2019)
- [A Modern Analysis of Hutchinson's Trace Estimator](https://mlsanity.com/paper/2012.12895/) (2020)
- [Multiway p-spectral graph cuts on Grassmann manifolds](https://mlsanity.com/paper/2008.13210/) (2020)

## Abstract (arXiv)

The real-life data have a complex and non-linear structure due to their nature. These non-linearities and the large number of features can usually cause problems such as the empty-space phenomenon and the well-known curse of dimensionality. Finding the nearly optimal representation of the dataset in a lower-dimensional space (i.e. dimensionality reduction) offers an applicable mechanism for improving the success of machine learning tasks. However, estimating the required data dimension for the nearly optimal representation (intrinsic dimension) can be very costly, particularly if one deals with big data. We propose a highly efficient and robust intrinsic dimension estimation approach that only relies on matrix-vector products for dimensionality reduction methods. An experimental study is also conducted to compare the performance of proposed method with state of the art approaches.