# Calibrating dimension reduction hyperparameters in the presence of noise

arXiv: https://arxiv.org/abs/2312.02946 · HTML: https://mlsanity.com/paper/2312.02946/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 31 references that are in the map.
- First posted on arXiv 5 Dec 2023.

## Builds on (most cited first, 6 of 6)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization](https://mlsanity.com/paper/2012.04456/) (2020)
- [TriMap: Large-scale Dimensionality Reduction Using Triplets](https://mlsanity.com/paper/1910.00204/) (2019)
- [Automatic Selection of t-SNE Perplexity](https://mlsanity.com/paper/1708.03229/) (2017)
- [Perplexity-free Parametric t-SNE](https://mlsanity.com/paper/2010.01359/) (2020)
- [Navigating Perplexity: A linear relationship with the data set size in t-SNE embeddings](https://mlsanity.com/paper/2308.15513/) (2023)

## Built on by (most cited first, 1 of 1)

- [Local Neighborhood Instability in Parametric Projections: Quantitative and Visual Analysis](https://mlsanity.com/paper/2604.21617/) (2026)

## Abstract (arXiv)

The goal of dimension reduction tools is to construct a low-dimensional representation of high-dimensional data. These tools are employed for a variety of reasons such as noise reduction, visualization, and to lower computational costs. However, there is a fundamental issue that is discussed in other modeling problems that is often overlooked in dimension reduction -- overfitting. In the context of other modeling problems, techniques such as feature-selection, cross-validation, and regularization are employed to combat overfitting, but rarely are such precautions taken when applying dimension reduction. Prior applications of the two most popular non-linear dimension reduction methods, t-SNE and UMAP, fail to acknowledge data as a combination of signal and noise when assessing performance. These methods are typically calibrated to capture the entirety of the data, not just the signal. In this paper, we demonstrate the importance of acknowledging noise when calibrating hyperparameters and present a framework that enables users to do so. We use this framework to explore the role hyperparameter calibration plays in overfitting the data when applying t-SNE and UMAP. More specifically, we show previously recommended values for perplexity and n_neighbors are too small and overfit the noise. We also provide a workflow others may use to calibrate hyperparameters in the presence of noise.