# Dimensionality Reduction as Probabilistic Inference

arXiv: https://arxiv.org/abs/2304.07658 · HTML: https://mlsanity.com/paper/2304.07658/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 44 references that are in the map.
- First posted on arXiv 15 Apr 2023.

## Builds on (most cited first, 8 of 8)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Mat'ern Gaussian Processes on Graphs](https://mlsanity.com/paper/2010.15538/) (2020)
- [Bayesian Semi-supervised Learning with Graph Gaussian Processes](https://mlsanity.com/paper/1809.04379/) (2018)
- [Graph Convolutional Gaussian Processes For Link Prediction](https://mlsanity.com/paper/2002.04337/) (2020)

## Built on by (most cited first, 4 of 4)

- [The Shape of Attraction in UMAP: Exploring the Embedding Forces in Dimensionality Reduction](https://mlsanity.com/paper/2503.09101/) (2025)
- [Uncertainty as a Predictor: Leveraging Self-Supervised Learning for Zero-Shot MOS Prediction](https://mlsanity.com/paper/2312.15616/) (2023)
- [Towards One Model for Classical Dimensionality Reduction: A Probabilistic Perspective on UMAP and t-SNE](https://mlsanity.com/paper/2405.17412/) (2024)
- [Transformers as Unrolled Inference in Probabilistic Laplacian Eigenmaps: An Interpretation and Potential Improvements](https://mlsanity.com/paper/2507.21040/) (2025)

## Abstract (arXiv)

Dimensionality reduction (DR) algorithms compress high-dimensional data into a lower dimensional representation while preserving important features of the data. DR is a critical step in many analysis pipelines as it enables visualisation, noise reduction and efficient downstream processing of the data. In this work, we introduce the ProbDR variational framework, which interprets a wide range of classical DR algorithms as probabilistic inference algorithms in this framework. ProbDR encompasses PCA, CMDS, LLE, LE, MVU, diffusion maps, kPCA, Isomap, (t-)SNE, and UMAP. In our framework, a low-dimensional latent variable is used to construct a covariance, precision, or a graph Laplacian matrix, which can be used as part of a generative model for the data. Inference is done by optimizing an evidence lower bound. We demonstrate the internal consistency of our framework and show that it enables the use of probabilistic programming languages (PPLs) for DR. Additionally, we illustrate that the framework facilitates reasoning about unseen data and argue that our generative models approximate Gaussian processes (GPs) on manifolds. By providing a unified view of DR, our framework facilitates communication, reasoning about uncertainties, model composition, and extensions, particularly when domain knowledge is present.