# Topologically Regularized Data Embeddings

arXiv: https://arxiv.org/abs/2110.09193 · HTML: https://mlsanity.com/paper/2110.09193/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 66 references that are in the map.
- First posted on arXiv 18 Oct 2021.

## Builds on (most cited first, 15 of 15)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [SimplE Embedding for Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/1802.04868/) (2018)
- [Graph Embedding Techniques, Applications, and Performance: A Survey](https://mlsanity.com/paper/1705.02801/) (2017)
- [Cooperative Game Theory Approaches for Network Partitioning](https://mlsanity.com/paper/1707.03587/) (2017)
- [A roadmap for the computation of persistent homology](https://mlsanity.com/paper/1506.08903/) (2015)
- [Topological Autoencoders](https://mlsanity.com/paper/1906.00722/) (2019)
- [Neural Collaborative Filtering vs. Matrix Factorization Revisited](https://mlsanity.com/paper/2005.09683/) (2020)
- [Pymanopt: A Python Toolbox for Optimization on Manifolds using Automatic Differentiation](https://mlsanity.com/paper/1603.03236/) (2016)
- [A Topology Layer for Machine Learning](https://mlsanity.com/paper/1905.12200/) (2019)
- [A Topological Regularizer for Classifiers via Persistent Homology](https://mlsanity.com/paper/1806.10714/) (2018)
- [A Fast and Robust Method for Global Topological Functional Optimization](https://mlsanity.com/paper/2009.08496/) (2020)
- [Improving Metric Dimensionality Reduction with Distributed Topology](https://mlsanity.com/paper/2106.07613/) (2021)
- [The Curse Revisited: When are Distances Informative for the Ground Truth in Noisy High-Dimensional Data?](https://mlsanity.com/paper/2109.10569/) (2021)

## Built on by (most cited first, 5 of 5)

- [Differentiable Euler Characteristic Transforms for Shape Classification](https://mlsanity.com/paper/2310.07630/) (2023)
- [TopoMap++: A faster and more space efficient technique to compute projections with topological guarantees](https://mlsanity.com/paper/2409.07257/) (2024)
- [Topological Inductive Bias fosters Multiple Instance Learning in Data-Scarce Scenarios](https://mlsanity.com/paper/2307.14025/) (2023)
- [GroupEnc: encoder with group loss for global structure preservation](https://mlsanity.com/paper/2309.02917/) (2023)
- [Lens functions for exploring UMAP Projections with Domain Knowledge](https://mlsanity.com/paper/2405.09204/) (2024)

## Abstract (arXiv)

Unsupervised feature learning often finds low-dimensional embeddings that capture the structure of complex data. For tasks for which prior expert topological knowledge is available, incorporating this into the learned representation may lead to higher quality embeddings. For example, this may help one to embed the data into a given number of clusters, or to accommodate for noise that prevents one from deriving the distribution of the data over the model directly, which can then be learned more effectively. However, a general tool for integrating different prior topological knowledge into embeddings is lacking. Although differentiable topology layers have been recently developed that can (re)shape embeddings into prespecified topological models, they have two important limitations for representation learning, which we address in this paper. First, the currently suggested topological losses fail to represent simple models such as clusters and flares in a natural manner. Second, these losses neglect all original structural (such as neighborhood) information in the data that is useful for learning. We overcome these limitations by introducing a new set of topological losses, and proposing their usage as a way for topologically regularizing data embeddings to naturally represent a prespecified model. We include thorough experiments on synthetic and real data that highlight the usefulness and versatility of this approach, with applications ranging from modeling high-dimensional single-cell data, to graph embedding.