# Measuring the Validity of Clustering Validation Datasets

arXiv: https://arxiv.org/abs/2503.01097 · HTML: https://mlsanity.com/paper/2503.01097/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 70 references that are in the map.
- First posted on arXiv 3 Mar 2025.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Understanding Dataset Difficulty with V-Usable Information](https://mlsanity.com/paper/2110.08420/) (2021)
- [Discovering Latent Concepts Learned in BERT](https://mlsanity.com/paper/2205.07237/) (2022)
- [Unsupervised User Stance Detection on Twitter](https://mlsanity.com/paper/1904.02000/) (2019)
- [A better measure of relative prediction accuracy for model selection and model estimation](https://mlsanity.com/paper/2105.05249/) (2021)
- [Unveiling High-dimensional Backstage: A Survey for Reliable Visual Analytics with Dimensionality Reduction](https://mlsanity.com/paper/2501.10168/) (2025)
- [Classes are not Clusters: Improving Label-based Evaluation of Dimensionality Reduction](https://mlsanity.com/paper/2308.00278/) (2023)
- [From A-to-Z Review of Clustering Validation Indices](https://mlsanity.com/paper/2407.20246/) (2024)
- [CLAMS: A Cluster Ambiguity Measure for Estimating Perceptual Variability in Visual Clustering](https://mlsanity.com/paper/2308.00284/) (2023)
- [Multi-label Classification via Adaptive Resonance Theory-based Clustering](https://mlsanity.com/paper/2103.01511/) (2021)
- [ClustML: A Measure of Cluster Pattern Complexity in Scatterplots Learnt from Human-labeled Groupings](https://mlsanity.com/paper/2106.00599/) (2021)

## Built on by (most cited first, 6 of 6)

- [Metric Design != Metric Behavior: Improving Metric Selection for the Unbiased Evaluation of Dimensionality Reduction](https://mlsanity.com/paper/2507.02225/) (2025)
- [Dataset-Adaptive Dimensionality Reduction](https://mlsanity.com/paper/2507.11984/) (2025)
- [Understanding Bias in Perceiving Dimensionality Reduction Projections](https://mlsanity.com/paper/2507.20805/) (2025)
- [CLUBench: A Clustering Benchmark](https://mlsanity.com/paper/2605.29933/) (2026)
- [Why Can't I See My Clusters? A Precision-Recall Approach to Dimensionality Reduction Validation](https://mlsanity.com/paper/2509.04222/) (2025)
- [Automatic depth-based local center clustering via β-integrated local depth and adaptive grouping](https://mlsanity.com/paper/2609.26748/) (2026)

## Abstract (arXiv)

Clustering techniques are often validated using benchmark datasets where class labels are used as ground-truth clusters. However, depending on the datasets, class labels may not align with the actual data clusters, and such misalignment hampers accurate validation. Therefore, it is essential to evaluate and compare datasets regarding their cluster-label matching (CLM), i.e., how well their class labels match actual clusters. Internal validation measures (IVMs), like Silhouette, can compare CLM over different labeling of the same dataset, but are not designed to do so across different datasets. We thus introduce Adjusted IVMs as fast and reliable methods to evaluate and compare CLM across datasets. We establish four axioms that require validation measures to be independent of data properties not related to cluster structure (e.g., dimensionality, dataset size). Then, we develop standardized protocols to convert any IVM to satisfy these axioms, and use these protocols to adjust six widely used IVMs. Quantitative experiments (1) verify the necessity and effectiveness of our protocols and (2) show that adjusted IVMs outperform the competitors, including standard IVMs, in accurately evaluating CLM both within and across datasets. We also show that the datasets can be filtered or improved using our method to form more reliable benchmarks for clustering validation.