# Classes are not Clusters: Improving Label-based Evaluation of Dimensionality Reduction

arXiv: https://arxiv.org/abs/2308.00278 · HTML: https://mlsanity.com/paper/2308.00278/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 78 references that are in the map.
- First posted on arXiv 1 Aug 2023.

## Builds on (most cited first, 13 of 13)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Topological Autoencoders](https://mlsanity.com/paper/1906.00722/) (2019)
- [Revisiting Dimensionality Reduction Techniques for Visual Cluster Analysis: An Empirical Study](https://mlsanity.com/paper/2110.02894/) (2021)
- [Robust Topological Inference: Distance To a Measure and Kernel Distance](https://mlsanity.com/paper/1412.7197/) (2014)
- [Measuring and Explaining the Inter-Cluster Reliability of Multidimensional Projections](https://mlsanity.com/paper/2107.07859/) (2021)
- [Automatic Selection of t-SNE Perplexity](https://mlsanity.com/paper/1708.03229/) (2017)
- [Uniform Manifold Approximation with Two-phase Optimization](https://mlsanity.com/paper/2205.00420/) (2022)
- [Modeling the Influence of Visual Density on Cluster Perception in Scatterplots Using Topology](https://mlsanity.com/paper/2007.13872/) (2020)
- [Automatic Scatterplot Design Optimization for Clustering Identification](https://mlsanity.com/paper/2207.03355/) (2022)
- [Sanity Check for External Clustering Validation Benchmarks using Internal Validation Measures](https://mlsanity.com/paper/2209.10042/) (2022)
- [Distortion-Aware Brushing for Reliable Cluster Analysis in Multidimensional Projections](https://mlsanity.com/paper/2201.06379/) (2022)
- [ClassSPLOM -- A Scatterplot Matrix to Visualize Separation of Multiclass Multidimensional Data](https://mlsanity.com/paper/2201.12822/) (2022)

## Built on by (most cited first, 11 of 11)

- [Stop Misusing t-SNE and UMAP for Visual Analytics](https://mlsanity.com/paper/2506.08725/) (2025)
- [Measuring the Validity of Clustering Validation Datasets](https://mlsanity.com/paper/2503.01097/) (2025)
- [Dataset-Adaptive Dimensionality Reduction](https://mlsanity.com/paper/2507.11984/) (2025)
- [UMATO: Bridging Local and Global Structures for Reliable Visual Analytics with Dimensionality Reduction](https://mlsanity.com/paper/2508.16227/) (2025)
- [How Scale Breaks "Normalized Stress" and KL Divergence: Rethinking Quality Metrics](https://mlsanity.com/paper/2510.08660/) (2025)
- [MAPLE: Self-Supervised Learning-Enhanced Nonlinear Dimensionality Reduction for Visual Analysis](https://mlsanity.com/paper/2601.20173/) (2026)
- [Standardness Clouds Meaning: A Position Regarding the Informed Usage of Standard Datasets](https://mlsanity.com/paper/2406.13552/) (2024)
- [Navigating High-Dimensional Backstage: A Guide for Exploring Literature for the Reliable Use of Dimensionality Reduction](https://mlsanity.com/paper/2506.14820/) (2025)
- [Why Can't I See My Clusters? A Precision-Recall Approach to Dimensionality Reduction Validation](https://mlsanity.com/paper/2509.04222/) (2025)
- [A Spectral Decomposition Framework for Multiscale Nonlinear Dimensionality Reduction](https://mlsanity.com/paper/2604.02535/) (2026)
- [Class Angular Distortion Index for Dimensionality Reduction](https://mlsanity.com/paper/2605.00637/) (2026)

## Abstract (arXiv)

A common way to evaluate the reliability of dimensionality reduction (DR) embeddings is to quantify how well labeled classes form compact, mutually separated clusters in the embeddings. This approach is based on the assumption that the classes stay as clear clusters in the original high-dimensional space. However, in reality, this assumption can be violated; a single class can be fragmented into multiple separated clusters, and multiple classes can be merged into a single cluster. We thus cannot always assure the credibility of the evaluation using class labels. In this paper, we introduce two novel quality measures -- Label-Trustworthiness and Label-Continuity (Label-T&C) -- advancing the process of DR evaluation based on class labels. Instead of assuming that classes are well-clustered in the original space, Label-T&C work by (1) estimating the extent to which classes form clusters in the original and embedded spaces and (2) evaluating the difference between the two. A quantitative evaluation showed that Label-T&C outperform widely used DR evaluation measures (e.g., Trustworthiness and Continuity, Kullback-Leibler divergence) in terms of the accuracy in assessing how well DR embeddings preserve the cluster structure, and are also scalable. Moreover, we present case studies demonstrating that Label-T&C can be successfully used for revealing the intrinsic characteristics of DR techniques and their hyperparameters.