mlsanityData to 14 Oct 2026

The Shape of Attraction in UMAP: Exploring the Embedding Forces in Dimensionality Reduction

arXiv 2503.09101 ↗First posted 12 Mar 2025cs.LG cs.AI cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Uniform manifold approximation and projection (UMAP) is among the most popular neighbor embedding methods. The method samples pairs of point indices according to similarities in the high-dimensional space, and applies attractive and repulsive forces to their coordinates in the low-dimensional embedding. In this paper, we analyze the forces to reveal their effects on cluster formations and visualization, and compare UMAP to its contemporaries. Repulsion emphasizes differences, controlling cluster boundaries and inter-cluster distance. Attraction is more subtle, as attractive tension between points can manifest simultaneously as attraction and repulsion in the lower-dimensional mapping. This explains the need for learning rate annealing and motivates the different treatments between attractive and repulsive terms. Moreover, by modifying attraction, we improve the consistency of cluster formation under random initialization. Overall, our analysis provides a mechanistic understanding of UMAP and related embedding methods.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 31 of its 64 references that are in the map. First posted on arXiv 12 Mar 2025.

Builds on

31 of 64 references.

References · 31

  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  2. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  3. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  4. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  5. 2024The Platonic Representation Hypothesiscited by 382
  6. 2020Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualizationcited by 173
  7. 2020Parametric UMAP embeddings for representation and semi-supervised learningcited by 104
  8. 2016Visualizing Large-scale and High-dimensional Datacited by 79
  9. 2019TriMap: Large-scale Dimensionality Reduction Using Tripletscited by 60
  10. 2021Deep Neural Networks for Rank-Consistent Ordinal Regression Based On Conditional Probabilitiescited by 59
  11. 2021Theoretical Foundations of t-SNE for Visualizing High-Dimensional Clustered Datacited by 55
  12. 2017Clustering with t-SNE, provablycited by 51
19 more
  1. 2018An Analysis of the t-SNE Algorithm for Data Visualizationcited by 49
  2. 2020Attraction-Repulsion Spectrum in Neighbor Embeddingscited by 41
  3. 2021On UMAP's true loss functioncited by 32
  4. 2022Your Contrastive Learning Is Secretly Doing Stochastic Neighbor Embeddingcited by 28
  5. 2022From t-SNE to UMAP with contrastive learningcited by 28
  6. 2021Minimum-Distortion Embeddingcited by 26
  7. 2019Heavy-tailed kernels reveal a finer cluster structure in t-SNE visualisationscited by 22
  8. 2024Dimension Reduction with Locally Adjusted Graphscited by 9
  9. 2017Stochastic Neighbor Embedding separates well-separated clusterscited by 7
  10. 2022Manifold-aligned Neighbor Embeddingcited by 7
  11. 2022SQuadMDS: a lean Stochastic Quartet MDS improving global structure preservation in neighbor embedding like t-SNE and UMAPcited by 5
  12. 2023ActUp: Analyzing and Consolidating tSNE and UMAPcited by 5
  13. 2023Dimensionality Reduction as Probabilistic Inferencecited by 4
  14. 2022Interpretable Dimensionality Reduction by Feature Preserving Manifold Approximation and Projectioncited by 3
  15. 2023Relating tSNE and UMAP to Classical Dimensionality Reductioncited by 3
  16. 2024Towards One Model for Classical Dimensionality Reduction: A Probabilistic Perspective on UMAP and t-SNEcited by 3
  17. 2024Outlier Detection in Large Radiological Datasets using UMAPcited by 3
  18. 2023The Information Geometry of UMAPcited by 1
  19. 2024Exploring the Influence of Dimensionality Reduction on Anomaly Detection Performance in Multivariate Time Seriescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026MAPLE: Self-Supervised Learning-Enhanced Nonlinear Dimensionality Reduction for Visual Analysiscited by 1
  2. 2026Enhancing Language Models for Robust Greenwashing Detectioncited by 0
  3. 2026On Out-of-sample Embedding in UMAPcited by 0
  4. 2026CosMAP: Contrastive Manifold Approximation and Projection for Dimensionality Reduction of Omics and Genealogical Datacited by 0
  5. 2026ibUMAP: Coherent and Scalable Field Evaluation for UMAP Optimizationcited by 0