# Class Density and Dataset Quality in High-Dimensional, Unstructured Data

arXiv: https://arxiv.org/abs/2202.03856 · HTML: https://mlsanity.com/paper/2202.03856/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 24 references that are in the map.
- First posted on arXiv 8 Feb 2022.

## Builds on (most cited first, 7 of 7)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [EMNIST: an extension of MNIST to handwritten letters](https://mlsanity.com/paper/1702.05373/) (2017)
- [No Routing Needed Between Capsules](https://mlsanity.com/paper/2001.09136/) (2020)
- [Homogeneous Vector Capsules Enable Adaptive Gradient Descent in Convolutional Neural Networks](https://mlsanity.com/paper/1906.08676/) (2019)
- [Towards an Analytical Definition of Sufficient Data](https://mlsanity.com/paper/2202.03238/) (2022)
- [On the Importance of Capturing a Sufficient Diversity of Perspective for the Classification of micro-PCBs](https://mlsanity.com/paper/2101.11164/) (2021)

## Built on by (most cited first, 1 of 1)

- [A Dataset Fusion Algorithm for Generalised Anomaly Detection in Homogeneous Periodic Time Series Datasets](https://mlsanity.com/paper/2305.08197/) (2023)

## Abstract (arXiv)

We provide a definition for class density that can be used to measure the aggregate similarity of the samples within each of the classes in a high-dimensional, unstructured dataset. We then put forth several candidate methods for calculating class density and analyze the correlation between the values each method produces with the corresponding individual class test accuracies achieved on a trained model. Additionally, we propose a definition for dataset quality for high-dimensional, unstructured data and show that those datasets that met a certain quality threshold (experimentally demonstrated to be > 10 for the datasets studied) were candidates for eliding redundant data based on the individual class densities.