# Interpreting the Curse of Dimensionality from Distance Concentration and Manifold Effect

arXiv: https://arxiv.org/abs/2401.00422 · HTML: https://mlsanity.com/paper/2401.00422/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 30 references that are in the map.
- First posted on arXiv 31 Dec 2023.

## Builds on (most cited first, 8 of 8)

- [Learning in High Dimension Always Amounts to Extrapolation](https://mlsanity.com/paper/2110.09485/) (2021)
- [Overfitting Mechanism and Avoidance in Deep Neural Networks](https://mlsanity.com/paper/1901.06566/) (2019)
- [Supervised Dimensionality Reduction for Big Data](https://mlsanity.com/paper/1709.01233/) (2017)
- [The Curse Revisited: When are Distances Informative for the Ground Truth in Noisy High-Dimensional Data?](https://mlsanity.com/paper/2109.10569/) (2021)
- [Optimizing and accelerating space-time Ripley's K function based on Apache Spark for distributed spatiotemporal point pattern analysis](https://mlsanity.com/paper/1912.04753/) (2019)
- [MeanCut: A Greedy-Optimized Graph Clustering via Path-based Similarity and Degree Descent Criterion](https://mlsanity.com/paper/2312.04067/) (2023)
- [How I learned to stop worrying and love the curse of dimensionality: an appraisal of cluster validation in high-dimensional spaces](https://mlsanity.com/paper/2201.05214/) (2022)
- [A Robust and Efficient Boundary Point Detection Method by Measuring Local Direction Dispersion](https://mlsanity.com/paper/2312.04065/) (2023)

## Built on by (most cited first, 14 of 14)

- [In-Distribution Steering: Balancing Control and Coherence in Language Model Generation](https://mlsanity.com/paper/2510.13285/) (2025)
- [DINO-Tok: Adapting DINO for Visual Tokenizers](https://mlsanity.com/paper/2511.20565/) (2025)
- [Integrating Travel Behavior Forecasting and Generative Modeling for Predicting Future Urban Mobility and Spatial Transformations](https://mlsanity.com/paper/2503.21158/) (2025)
- [Sampling-enabled scalable manifold learning unveils the discriminative cluster structure of high-dimensional data](https://mlsanity.com/paper/2401.01100/) (2024)
- [Data clustering: a fundamental method in data science and management](https://mlsanity.com/paper/2412.18760/) (2024)
- [Deep Clustering Evaluation: How to Validate Internal Clustering Validation Measures](https://mlsanity.com/paper/2403.14830/) (2024)
- [Towards High Supervised Learning Utility Training Data Generation: Data Pruning and Column Reordering](https://mlsanity.com/paper/2507.10088/) (2025)
- [Approximate Diverse k-nearest Neighbor Search in Vector Database](https://mlsanity.com/paper/2510.27243/) (2025)
- [Value-Aware Product Recommendation by Customer Segmentation using a suitable High-Dimensional Similarity Measure](https://mlsanity.com/paper/2604.26983/) (2026)
- [Demographic Metadata as Construct-Irrelevant Noise in DistilBERT-Based Automated Essay Scoring](https://mlsanity.com/paper/2606.21066/) (2026)
- [Secret Scanner Agent: Extracting Secrets and Access Context from Unstructured Documents](https://mlsanity.com/paper/2607.09011/) (2026)
- [Measuring Distortion in the Empty Regions of Dimensionality Reduction Scatterplots with the Gap Index](https://mlsanity.com/paper/2607.28324/) (2026)
- [A Cascaded Unsupervised-Supervised NLP Pipeline for Detecting Accusatory Language in Public Procurement](https://mlsanity.com/paper/2608.12269/) (2026)
- [Don't Cut Corners: How Training Outside the Prior Makes Simulation-Based Inference More Robust](https://mlsanity.com/paper/2608.12470/) (2026)

## Abstract (arXiv)

The characteristics of data like distribution and heterogeneity, become more complex and counterintuitive as dimensionality increases. This phenomenon is known as curse of dimensionality, where common patterns and relationships (e.g., internal pattern and boundary pattern) that hold in low-dimensional space may be invalid in higher-dimensional space. It leads to a decreasing performance for the regression, classification, or clustering models or algorithms. Curse of dimensionality can be attributed to many causes. In this paper, we first summarize the potential challenges associated with manipulating high-dimensional data, and explains the possible causes for the failure of regression, classification, or clustering tasks. Subsequently, we delve into two major causes of the curse of dimensionality, distance concentration, and manifold effect, by performing theoretical and empirical analyses. The results demonstrate that, as the dimensionality increases, nearest neighbor search (NNS) using three classical distance measurements, Minkowski distance, Chebyshev distance, and cosine distance, becomes meaningless. Meanwhile, the data incorporates more redundant features, and the variance contribution of principal component analysis (PCA) is skewed towards a few dimensions.