# Metric Design != Metric Behavior: Improving Metric Selection for the Unbiased Evaluation of Dimensionality Reduction

arXiv: https://arxiv.org/abs/2507.02225 · HTML: https://mlsanity.com/paper/2507.02225/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 37 references that are in the map.
- First posted on arXiv 3 Jul 2025.

## Builds on (most cited first, 15 of 15)

- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization](https://mlsanity.com/paper/2012.04456/) (2020)
- [Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics](https://mlsanity.com/paper/2006.06264/) (2020)
- [Topological Autoencoders](https://mlsanity.com/paper/1906.00722/) (2019)
- [ROUGE 2.0: Updated and Improved Measures for Evaluation of Summarization Tasks](https://mlsanity.com/paper/1803.01937/) (2018)
- [TriMap: Large-scale Dimensionality Reduction Using Triplets](https://mlsanity.com/paper/1910.00204/) (2019)
- [Towards a Guideline for Evaluation Metrics in Medical Image Segmentation](https://mlsanity.com/paper/2202.05273/) (2022)
- [Unveiling High-dimensional Backstage: A Survey for Reliable Visual Analytics with Dimensionality Reduction](https://mlsanity.com/paper/2501.10168/) (2025)
- [A Visual Interaction Framework for Dimensionality Reduction Based Data Exploration](https://mlsanity.com/paper/1811.12199/) (2018)
- [ZADU: A Python Library for Evaluating the Reliability of Dimensionality Reduction Embeddings](https://mlsanity.com/paper/2308.00282/) (2023)
- [Uniform Manifold Approximation with Two-phase Optimization](https://mlsanity.com/paper/2205.00420/) (2022)
- [Large-Scale Evaluation of Topic Models and Dimensionality Reduction Methods for 2D Text Spatialization](https://mlsanity.com/paper/2307.11770/) (2023)
- [Measuring the Validity of Clustering Validation Datasets](https://mlsanity.com/paper/2503.01097/) (2025)
- [A Large-Scale Sensitivity Analysis on Latent Embeddings and Dimensionality Reductions for Text Spatializations](https://mlsanity.com/paper/2407.17876/) (2024)
- [On the cohesion and separability of average-link for hierarchical agglomerative clustering](https://mlsanity.com/paper/2411.05097/) (2024)

## Built on by (most cited first, 2 of 2)

- [Class Angular Distortion Index for Dimensionality Reduction](https://mlsanity.com/paper/2605.00637/) (2026)
- [A Fixed-Radius Distance-Band Benchmark for Dimensionality-Reduction Fidelity](https://mlsanity.com/paper/2608.21779/) (2026)

## Abstract (arXiv)

Evaluating the accuracy of dimensionality reduction (DR) projections in preserving the structure of high-dimensional data is crucial for reliable visual analytics. Diverse evaluation metrics targeting different structural characteristics have thus been developed. However, evaluations of DR projections can become biased if highly correlated metrics--those measuring similar structural characteristics--are inadvertently selected, favoring DR techniques that emphasize those characteristics. To address this issue, we propose a novel workflow that reduces bias in the selection of evaluation metrics by clustering metrics based on their empirical correlations rather than on their intended design characteristics alone. Our workflow works by computing metric similarity using pairwise correlations, clustering metrics to minimize overlap, and selecting a representative metric from each cluster. Quantitative experiments demonstrate that our approach improves the stability of DR evaluation, which indicates that our workflow contributes to mitigating evaluation bias.