mlsanityData to 7 Oct 2026

Estimating the intrinsic dimension of datasets by a minimal neighborhood information

arXiv 1803.06992 ↗First posted 19 Mar 2018stat.ML cs.LGcited by 246

Not on the map. Why?

Abstract (arXiv)

Analyzing large volumes of high-dimensional data is an issue of fundamental importance in data science, molecular simulations and beyond. Several approaches work on the assumption that the important content of a dataset belongs to a manifold whose Intrinsic Dimension (ID) is much lower than the crude large number of coordinates. Such manifold is generally twisted and curved, in addition points on it will be non-uniformly distributed: two factors that make the identification of the ID and its exploitation really hard. Here we propose a new ID estimator using only the distance of the first and the second nearest neighbor of each point in the sample. This extreme minimality enables us to reduce the effects of curvature, of density variation, and the resulting computational cost. The ID estimator is theoretically exact in uniformly distributed datasets, and provides consistent measures in general. When used in combination with block analysis, it allows discriminating the relevant dimensions as a function of the block size. This allows estimating the ID even when the data lie on a manifold perturbed by a high-dimensional noise, a situation often encountered in real world data sets. We demonstrate the usefulness of the approach on molecular simulations and image analysis.

Key facts. Cited by 246 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 25; 2023: 21; 2024: 34; 2025: 51; 2026: 98.

246 papers.

Sort:

By area:

  1. 2021The Intrinsic Dimension of Images and Its Impact on Learningcited by 307
  2. 2021Dataset Distillation with Infinitely Wide Convolutional Networkscited by 215
  3. 2021A Universal Law of Robustness via Isoperimetrycited by 150
  4. 2023Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMscited by 121
  5. 2023Intrinsic Dimension Estimation for Robust Detection of AI-Generated Textscited by 111
  6. 2023The geometry of hidden representations of large transformer modelscited by 108
  7. 2022A Solvable Model of Neural Scaling Lawscited by 101
  8. 2025EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modelingcited by 85
  9. 2021Intrinsic Dimension, Persistent Homology and Generalization in Neural Networkscited by 70
  10. 2021Scikit-dimension: a Python package for intrinsic dimension estimationcited by 68
  11. 2024Emergence of a High-Dimensional Abstraction Phase in Language Transformerscited by 56
  12. 2024Characterizing Truthfulness in Large Language Model Generations with Local Intrinsic Dimensioncited by 45
38 more
  1. 2024A Geometric View of Data Complexity: Efficient Local Intrinsic Dimension Estimation with Diffusion Modelscited by 41
  2. 2024Loss Landscape Degeneracy and Stagewise Development in Transformerscited by 39
  3. 2021Rethinking Graph Auto-Encoder Models for Attributed Graph Clusteringcited by 37
  4. 2023The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Modelscited by 37
  5. 2023Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grokcited by 31
  6. 2022LIDL: Local Intrinsic Dimension Estimation Using Approximate Likelihoodcited by 30
  7. 2023Universality laws for Gaussian mixtures in generalized linear modelscited by 28
  8. 2023Bridging Information-Theoretic and Geometric Compression in Language Modelscited by 26
  9. 2021Probing transfer learning with a model of synthetic correlated datasetscited by 24
  10. 2022DADApy: Distance-based Analysis of DAta-manifolds in Pythoncited by 22
  11. 2024A Geometric Explanation of the Likelihood OOD Detection Paradoxcited by 22
  12. 2024Hardness of Learning Neural Networks under the Manifold Hypothesiscited by 18
  13. 2025Latent Thinking Optimization: Your Latent Reasoning Language Model Secretly Encodes Reward Signals in Its Latent Thoughtscited by 18
  14. 2025The Intrinsic Dimension of Prompts in Internal Representations of Large Language Modelscited by 17
  15. 2024Geometric Signatures of Compositionality Across a Language Model's Lifetimecited by 14
  16. 2025Token embeddings violate the manifold hypothesiscited by 14
  17. 2024The Effect of Intrinsic Dataset Properties on Generalization: Unraveling Learning Differences Between Natural and Medical Imagescited by 13
  18. 2024Evidence from fMRI Supports a Two-Phase Abstraction Process in Language Modelscited by 13
  19. 2022Topology and geometry of data manifold in deep learningcited by 12
  20. 2023Stable Anisotropic Regularizationcited by 12
  21. 2024Generalization Error Guaranteed Auto-Encoder-Based Nonlinear Model Reduction for Operator Learningcited by 12
  22. 2025LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformerscited by 12
  23. 2025Universally Converging Representations of Matter Across Scientific Foundation Modelscited by 12
  24. 2021Density estimation on low-dimensional manifolds: an inflation-deflation approachcited by 11
  25. 2023Representation Learning via Manifold Flattening and Reconstructioncited by 11
  26. 2023Data Representations' Study of Latent Image Manifoldscited by 11
  27. 2024Beyond Scalars: Concept-Based Alignment Analysis in Vision Transformerscited by 11
  28. 2021Reversible Gromov-Monge Sampler for Simulation-Based Inferencecited by 10
  29. 2022The Intrinsic Manifolds of Radiological Images and their Role in Deep Learningcited by 10
  30. 2025Are LLMs effective psychological assessors? Leveraging adaptive RAG for interpretable mental health screening through psychometric practicecited by 10
  31. 2025Redundancy, Isotropy, and Intrinsic Dimensionality of Prompt-based Text Embeddingscited by 10
  32. 2025Measuring the Intrinsic Dimension of Earth Representationscited by 10
  33. 2022Relating Regularization and Generalization through the Intrinsic Dimension of Activationscited by 9
  34. 2025Geometry of Decision Making in Language Modelscited by 8
  35. 2023Exploring the Representation Manifolds of Stable Diffusion Through the Lens of Intrinsic Dimensioncited by 7
  36. 2024GeLoRA: Geometric Adaptive Ranks For Efficient LoRA Fine-tuningcited by 7
  37. 2025Rethinking Predictive Modeling for LLM Routing: When Simple kNN Beats Complex Learned Routerscited by 7
  38. 2021A Theoretical-Empirical Approach to Estimating Sample Complexity of DNNscited by 6

50 most cited of 246 All 246 (JSON)