# A framework for benchmarking clustering algorithms

arXiv: https://arxiv.org/abs/2209.09493 · HTML: https://mlsanity.com/paper/2209.09493/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 36 references that are in the map.
- First posted on arXiv 20 Sep 2022.

## Builds on (most cited first, 4 of 4)

- [Are Cluster Validity Measures (In)valid?](https://mlsanity.com/paper/2208.01261/) (2022)
- [Validation of cluster analysis results on validation data: A systematic framework](https://mlsanity.com/paper/2103.01281/) (2021)
- [Genie: A new, fast, and outlier-resistant hierarchical clustering algorithm](https://mlsanity.com/paper/2209.05757/) (2022)
- [Normalised clustering accuracy: An asymmetric external cluster validity measure](https://mlsanity.com/paper/2209.02935/) (2022)

## Built on by (most cited first, 15 of 15)

- [Clustering with minimum spanning trees: How good can it be?](https://mlsanity.com/paper/2303.05679/) (2023)
- [AuToMATo: An Out-Of-The-Box Persistence-Based Clustering Algorithm](https://mlsanity.com/paper/2408.06958/) (2024)
- [On the Use of Relative Validity Indices for Comparing Clustering Approaches](https://mlsanity.com/paper/2404.10351/) (2024)
- [Feature graphs for interpretable unsupervised tree ensembles: centrality, interaction, and application in disease subtyping](https://mlsanity.com/paper/2404.17886/) (2024)
- [MNIST-Nd: a set of naturalistic datasets to benchmark clustering across dimensions](https://mlsanity.com/paper/2410.16124/) (2024)
- [Cluster Analysis with Resampling for Validation and Exploration (CARVE)](https://mlsanity.com/paper/2606.00327/) (2026)
- [Hierarchical clustering with OWA-based linkages, the Lance-Williams formula, and dendrogram inversions](https://mlsanity.com/paper/2303.05683/) (2023)
- [MMM and MMMSynth: Clustering of heterogeneous tabular data, and synthetic data generation](https://mlsanity.com/paper/2310.19454/) (2023)
- [Model-free Estimation of Latent Structure via Multiscale Nonparametric Maximum Likelihood](https://mlsanity.com/paper/2410.22248/) (2024)
- [Local Cluster Cardinality Estimation for Adaptive Mean Shift](https://mlsanity.com/paper/2508.12450/) (2025)
- [Khatri-Rao Clustering for Data Summarization](https://mlsanity.com/paper/2603.06602/) (2026)
- [The Mass Agreement Score: A Point-centric Measure of Cluster Size Consistency](https://mlsanity.com/paper/2603.23581/) (2026)
- [Lumbermark: Resistant Clustering by Chopping Up Mutual Reachability Minimum Spanning Trees](https://mlsanity.com/paper/2604.07143/) (2026)
- [ClusBench: The Clustering Benchmark Data Resource You've All Been Waiting For (?)](https://mlsanity.com/paper/2606.10673/) (2026)
- [G-NAC: Graph Neural Automata Clustering via Emergent Domain Formation](https://mlsanity.com/paper/2609.24823/) (2026)

## Abstract (arXiv)

The evaluation of clustering algorithms can involve running them on a variety of benchmark problems, and comparing their outputs to the reference, ground-truth groupings provided by experts. Unfortunately, many research papers and graduate theses consider only a small number of datasets. Also, the fact that there can be many equally valid ways to cluster a given problem set is rarely taken into account. In order to overcome these limitations, we have developed a framework whose aim is to introduce a consistent methodology for testing clustering algorithms. Furthermore, we have aggregated, polished, and standardised many clustering benchmark dataset collections referred to across the machine learning and data mining literature, and included new datasets of different dimensionalities, sizes, and cluster types. An interactive datasets explorer, the documentation of the Python API, a description of the ways to interact with the framework from other programming languages such as R or MATLAB, and other details are all provided at <https://clustering-benchmarks.gagolewski.com>.