mlsanityData to 7 Oct 2026

Scikit-dimension: a Python package for intrinsic dimension estimation

arXiv 2109.02596 ↗First posted 6 Sep 2021cs.LG stat.MLcited by 68

Not on the map. Why?

Abstract (arXiv)

Dealing with uncertainty in applications of machine learning to real-life data critically depends on the knowledge of intrinsic dimensionality (ID). A number of methods have been suggested for the purpose of estimating ID, but no standard package to easily apply them one by one or all at once has been implemented in Python. This technical note introduces textttscikit-dimension, an open-source Python package for intrinsic dimension estimation. textttscikit-dimension package provides a uniform implementation of most of the known ID estimators based on scikit-learn application programming interface to evaluate global and local intrinsic dimension, as well as generators of synthetic toy and benchmark datasets widespread in the literature. The package is developed with tools assessing the code quality, coverage, unit testing and continuous integration. We briefly describe the package and demonstrate its use in a large-scale (more than 500 datasets) benchmarking of methods for ID estimation in real-life and synthetic data. The source code is available from https://github.com/j-bac/scikit-dimension , the documentation is available from https://scikit-dimension.readthedocs.io .

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Builds on 17 of its 76 references that are in the map. First posted on arXiv 6 Sep 2021.

Builds on

17 of 76 references.

References · 17

  1. 2014OpenML: networked science in machine learningcited by 523
  2. 2018Estimating the intrinsic dimension of datasets by a minimal neighborhood informationcited by 246
  3. 2018To Trust Or Not To Trust A Classifiercited by 205
  4. 2018Blessing of dimensionality: mathematical foundations of the statistical physics of datacited by 55
  5. 2019Estimating the effective dimension of large biological datasets using Fisher separability analysiscited by 19
  6. 2018Correction of AI systems by linear discriminants: Probabilistic foundationscited by 17
  7. 2019Geometry-Aware Maximum Likelihood Estimation of Intrinsic Dimensioncited by 15
  8. 2020On Adversarial Examples and Stealth Attacks in Artificial Intelligence Systemscited by 13
  9. 2020Fractional norms and quasinorms do not help to overcome the curse of dimensionalitycited by 13
  10. 2020Trajectories, bifurcations and pseudotime in large clinical datasets: applications to myocardial infarction and diabetes datacited by 13
  11. 2021High-dimensional separability for one- and few-shot learningcited by 11
  12. 2018The unreasonable effectiveness of small neural ensembles in high-dimensional braincited by 10
5 more
  1. 2020General stochastic separation theorems with optimal boundscited by 8
  2. 2018Robust And Scalable Learning Of Complex Dataset Topologies Via Elpigraphcited by 5
  3. 2020Local intrinsic dimensionality estimators based on concentration of measurecited by 5
  4. 2020Rdimtools: An R package for Dimension Reduction and Intrinsic Dimension Estimationcited by 5
  5. 2021intRinsic: an R Package for Model-Based Estimation of the Intrinsic Dimension of a Datasetcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 14; 2024: 10; 2025: 21; 2026: 14.

68 papers.

Sort:

By area:

  1. 2023Intrinsic Dimension Estimation for Robust Detection of AI-Generated Textscited by 111
  2. 2023Multi-dimensional concept discovery (MCD): A unifying framework with completeness guaranteescited by 48
  3. 2024A Geometric View of Data Complexity: Efficient Local Intrinsic Dimension Estimation with Diffusion Modelscited by 41
  4. 2022LIDL: Local Intrinsic Dimension Estimation Using Approximate Likelihoodcited by 30
  5. 2024A Geometric Explanation of the Likelihood OOD Detection Paradoxcited by 22
  6. 2025The Intrinsic Dimension of Prompts in Internal Representations of Large Language Modelscited by 17
  7. 2022Topology and geometry of data manifold in deep learningcited by 12
  8. 2022The Intrinsic Manifolds of Radiological Images and their Role in Deep Learningcited by 10
  9. 2022A Measure of the Complexity of Neural Representations based on Partial Information Decompositioncited by 10
  10. 2023Simulation-based Inference for High-dimensional Data using Surjective Sequential Neural Likelihood Estimationcited by 8
  11. 2025Geometry of Decision Making in Language Modelscited by 8
  12. 2024GeLoRA: Geometric Adaptive Ranks For Efficient LoRA Fine-tuningcited by 7
38 more
  1. 2023Impact of time and note duration tokenizations on deep learning symbolic music modelingcited by 6
  2. 2023Relative intrinsic dimensionality is intrinsic to learningcited by 6
  3. 2024Robust estimation of the intrinsic dimension of data sets with quantum cognition machine learningcited by 6
  4. 2023Deep neural networks architectures from the perspective of manifold learningcited by 5
  5. 2024Analyzing Deep Transformer Models for Time Series Forecasting via Manifold Learningcited by 5
  6. 2022Sparse Subspace Clustering for Concept Discovery (SSCCD)cited by 4
  7. 2023Adversarial Estimation of Topological Dimension with Harmonic Score Mapscited by 4
  8. 2025A Connection Between Score Matching and Local Intrinsic Dimensioncited by 4
  9. 2025From Topology to Retrieval: Decoding Embedding Spaces with Unified Signaturescited by 4
  10. 2022Impact of dataset size and long-term ECoG-based BCI usage on deep learning decoders performancecited by 3
  11. 2023Information-Ordered Bottlenecks for Adaptive Semantic Compressioncited by 3
  12. 2024Scale adaptive and robust intrinsic dimension estimation via optimal neighbourhood identificationcited by 3
  13. 2026Automatic Layer Selection for Hallucination Detectioncited by 3
  14. 2025On Convolutions, Intrinsic Dimension, and Diffusion Modelscited by 2
  15. 2022Quasi-orthogonality and intrinsic dimensions as measures of learning and generalisationcited by 1
  16. 2022Multi-Sample zeta-mixup: Richer, More Realistic Synthetic Samples from a p-Series Interpolantcited by 1
  17. 2023Autoencoders with Intrinsic Dimension Constraints for Learning Low Dimensional Image Representationscited by 1
  18. 2023Principal subbundles for dimension reductioncited by 1
  19. 2023Understanding the Structure of QM7b and QM9 Quantum Mechanical Datasets Using Unsupervised Learningcited by 1
  20. 2024Dimensionality-Aware Outlier Detection: Theoretical and Experimental Analysiscited by 1
  21. 2025Estimating Dataset Dimension via Singular Metrics under the Manifold Hypothesis: Application to Inverse Problemscited by 1
  22. 2025Dataset-Adaptive Dimensionality Reductioncited by 1
  23. 2025Intrinsic Dimension Estimating Autoencoder (IDEA) Using CancelOut Layer and a Projected Losscited by 1
  24. 2026Mutual information and task-relevant latent dimensionalitycited by 1
  25. 2026Local Hessian Spectral Filtering for Robust Intrinsic Dimension Estimationcited by 1
  26. 2022Domain Adaptation Principal Component Analysis: base linear method for learning with out-of-distribution datacited by 0
  27. 2023Exploring Representational Disparities Between Multilingual and Bilingual Translation Modelscited by 0
  28. 2023Frequency maps reveal the correlation between Adversarial Attacks and Implicit Biascited by 0
  29. 2023Indexing Irises by Intrinsic Dimensioncited by 0
  30. 2024Density Estimation via Binless Multidimensional Integrationcited by 0
  31. 2024Beyond Flatland: A Geometric Take on Matching Methods for Treatment Effect Estimationcited by 0
  32. 2024Contrastive Learning to Fine-Tune Feature Extraction Models for the Visual Cortexcited by 0
  33. 2025Data-Informed Model Complexity Metric for Optimizing Symbolic Regression Modelscited by 0
  34. 2025A Novel Approach for Intrinsic Dimension Estimationcited by 0
  35. 2025When fractional quasi p-norms concentratecited by 0
  36. 2025Attractor learning for spatiotemporally chaotic dynamical systems using echo state networks with transfer learningcited by 0
  37. 2025Improving Clustering on Occupational Text Data through Dimensionality Reductioncited by 0
  38. 2025A Survey and Comparative Evaluation of Intrinsic Dimension Estimators under the Manifold Hypothesiscited by 0

50 most cited of 68 All 68 (JSON)