# High dimensionality: The latest challenge to data analysis

arXiv: https://arxiv.org/abs/1902.04679 · HTML: https://mlsanity.com/paper/1902.04679/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Feb 2019.

## Built on by (most cited first, 2 of 2)

- [An adaptive ensemble filter for heavy-tailed distributions: tuning-free inflation and localization](https://mlsanity.com/paper/2310.08741/) (2023)
- [Multivariate Time Series Anomaly Detection in Industry 5.0](https://mlsanity.com/paper/2503.15946/) (2025)

## Abstract (arXiv)

The advent of modern technology, permitting the measurement of thousands of characteristics simultaneously, has given rise to floods of data characterized by many large or even huge datasets. This new paradigm presents extraordinary challenges to data analysis and the question arises: how can conventional data analysis methods, devised for moderate or small datasets, cope with the complexities of modern data? The case of high dimensional data is particularly revealing of some of the drawbacks. We look at the case where the number of characteristics measured in an object is at least the number of observed objects and conclude that this configuration leads to geometrical and mathematical oddities and is an insurmountable barrier for the direct application of traditional methodologies. If scientists are going to ignore fundamental mathematical results arrived at in this paper and blindly use software to analyze data, the results of their analyses may not be trustful, and the findings of their experiments may never be validated. That is why new methods together with the wise use of traditional approaches are essential to progress safely through the present reality.