# Factoring out prior knowledge from low-dimensional embeddings

arXiv: https://arxiv.org/abs/2103.01828 · HTML: https://mlsanity.com/paper/2103.01828/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 39 references that are in the map.
- First posted on arXiv 2 Mar 2021.

## Builds on (most cited first, 6 of 6)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Visualizing and Measuring the Geometry of BERT](https://mlsanity.com/paper/1906.02715/) (2019)
- [Efficient Algorithms for t-distributed Stochastic Neighborhood Embedding](https://mlsanity.com/paper/1712.09005/) (2017)
- [Interactive Visual Data Exploration with Subjective Feedback: An Information-Theoretic Approach](https://mlsanity.com/paper/1710.08167/) (2017)
- [Conditional t-SNE: Complementary t-SNE embeddings through factoring out prior information](https://mlsanity.com/paper/1905.10086/) (2019)
- [Tell Me Something I Don't Know: Randomization Strategies for Iterative Data Mining](https://mlsanity.com/paper/2006.09467/) (2020)

## Built on by (most cited first, 1 of 1)

- [Preserving local densities in low-dimensional embeddings](https://mlsanity.com/paper/2301.13732/) (2023)

## Abstract (arXiv)

Low-dimensional embedding techniques such as tSNE and UMAP allow visualizing high-dimensional data and therewith facilitate the discovery of interesting structure. Although they are widely used, they visualize data as is, rather than in light of the background knowledge we have about the data. What we already know, however, strongly determines what is novel and hence interesting. In this paper we propose two methods for factoring out prior knowledge in the form of distance matrices from low-dimensional embeddings. To factor out prior knowledge from tSNE embeddings, we propose JEDI that adapts the tSNE objective in a principled way using Jensen-Shannon divergence. To factor out prior knowledge from any downstream embedding approach, we propose CONFETTI, in which we directly operate on the input distance matrices. Extensive experiments on both synthetic and real world data show that both methods work well, providing embeddings that exhibit meaningful structure that would otherwise remain hidden.