# The Shape of Attraction in UMAP: Exploring the Embedding Forces in Dimensionality Reduction

arXiv: https://arxiv.org/abs/2503.09101 · HTML: https://mlsanity.com/paper/2503.09101/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 64 references that are in the map.
- First posted on arXiv 12 Mar 2025.

## Builds on (most cited first, 30 of 31)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [The Platonic Representation Hypothesis](https://mlsanity.com/paper/2405.07987/) (2024)
- [Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization](https://mlsanity.com/paper/2012.04456/) (2020)
- [Parametric UMAP embeddings for representation and semi-supervised learning](https://mlsanity.com/paper/2009.12981/) (2020)
- [Visualizing Large-scale and High-dimensional Data](https://mlsanity.com/paper/1602.00370/) (2016)
- [TriMap: Large-scale Dimensionality Reduction Using Triplets](https://mlsanity.com/paper/1910.00204/) (2019)
- [Deep Neural Networks for Rank-Consistent Ordinal Regression Based On Conditional Probabilities](https://mlsanity.com/paper/2111.08851/) (2021)
- [Theoretical Foundations of t-SNE for Visualizing High-Dimensional Clustered Data](https://mlsanity.com/paper/2105.07536/) (2021)
- [Clustering with t-SNE, provably](https://mlsanity.com/paper/1706.02582/) (2017)
- [An Analysis of the t-SNE Algorithm for Data Visualization](https://mlsanity.com/paper/1803.01768/) (2018)
- [Attraction-Repulsion Spectrum in Neighbor Embeddings](https://mlsanity.com/paper/2007.08902/) (2020)
- [On UMAP's true loss function](https://mlsanity.com/paper/2103.14608/) (2021)
- [Your Contrastive Learning Is Secretly Doing Stochastic Neighbor Embedding](https://mlsanity.com/paper/2205.14814/) (2022)
- [From t-SNE to UMAP with contrastive learning](https://mlsanity.com/paper/2206.01816/) (2022)
- [Minimum-Distortion Embedding](https://mlsanity.com/paper/2103.02559/) (2021)
- [Heavy-tailed kernels reveal a finer cluster structure in t-SNE visualisations](https://mlsanity.com/paper/1902.05804/) (2019)
- [Dimension Reduction with Locally Adjusted Graphs](https://mlsanity.com/paper/2412.15426/) (2024)
- [Stochastic Neighbor Embedding separates well-separated clusters](https://mlsanity.com/paper/1702.02670/) (2017)
- [Manifold-aligned Neighbor Embedding](https://mlsanity.com/paper/2205.11257/) (2022)
- [SQuadMDS: a lean Stochastic Quartet MDS improving global structure preservation in neighbor embedding like t-SNE and UMAP](https://mlsanity.com/paper/2202.12087/) (2022)
- [ActUp: Analyzing and Consolidating tSNE and UMAP](https://mlsanity.com/paper/2305.07320/) (2023)
- [Dimensionality Reduction as Probabilistic Inference](https://mlsanity.com/paper/2304.07658/) (2023)
- [Interpretable Dimensionality Reduction by Feature Preserving Manifold Approximation and Projection](https://mlsanity.com/paper/2211.09321/) (2022)
- [Relating tSNE and UMAP to Classical Dimensionality Reduction](https://mlsanity.com/paper/2306.11898/) (2023)
- [Towards One Model for Classical Dimensionality Reduction: A Probabilistic Perspective on UMAP and t-SNE](https://mlsanity.com/paper/2405.17412/) (2024)
- [Outlier Detection in Large Radiological Datasets using UMAP](https://mlsanity.com/paper/2407.21263/) (2024)
- [The Information Geometry of UMAP](https://mlsanity.com/paper/2309.01237/) (2023)

## Built on by (most cited first, 5 of 5)

- [MAPLE: Self-Supervised Learning-Enhanced Nonlinear Dimensionality Reduction for Visual Analysis](https://mlsanity.com/paper/2601.20173/) (2026)
- [Enhancing Language Models for Robust Greenwashing Detection](https://mlsanity.com/paper/2601.21722/) (2026)
- [On Out-of-sample Embedding in UMAP](https://mlsanity.com/paper/2606.04451/) (2026)
- [CosMAP: Contrastive Manifold Approximation and Projection for Dimensionality Reduction of Omics and Genealogical Data](https://mlsanity.com/paper/2608.11269/) (2026)
- [ibUMAP: Coherent and Scalable Field Evaluation for UMAP Optimization](https://mlsanity.com/paper/2610.01445/) (2026)

## Abstract (arXiv)

Uniform manifold approximation and projection (UMAP) is among the most popular neighbor embedding methods. The method samples pairs of point indices according to similarities in the high-dimensional space, and applies attractive and repulsive forces to their coordinates in the low-dimensional embedding. In this paper, we analyze the forces to reveal their effects on cluster formations and visualization, and compare UMAP to its contemporaries. Repulsion emphasizes differences, controlling cluster boundaries and inter-cluster distance. Attraction is more subtle, as attractive tension between points can manifest simultaneously as attraction and repulsion in the lower-dimensional mapping. This explains the need for learning rate annealing and motivates the different treatments between attractive and repulsive terms. Moreover, by modifying attraction, we improve the consistency of cluster formation under random initialization. Overall, our analysis provides a mechanistic understanding of UMAP and related embedding methods.