# MEDAL: Manifold Embedding Distillation via Autoencoder Learning

arXiv: https://arxiv.org/abs/2605.24244 · HTML: https://mlsanity.com/paper/2605.24244/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 110 references that are in the map.
- First posted on arXiv 22 May 2026.

## Builds on (most cited first, 22 of 22)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization](https://mlsanity.com/paper/2012.04456/) (2020)
- [Recent Advances in Autoencoder-Based Representation Learning](https://mlsanity.com/paper/1812.05069/) (2018)
- [Approximating Continuous Functions by ReLU Nets of Minimal Width](https://mlsanity.com/paper/1710.11278/) (2017)
- [Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity](https://mlsanity.com/paper/1810.07770/) (2018)
- [Manifold learning: what, how, and why](https://mlsanity.com/paper/2311.03757/) (2023)
- [From Principal Subspaces to Principal Components with Linear Autoencoders](https://mlsanity.com/paper/1804.10253/) (2018)
- [Expressivity of Deep Neural Networks](https://mlsanity.com/paper/2007.04759/) (2020)
- [Automatic Selection of t-SNE Perplexity](https://mlsanity.com/paper/1708.03229/) (2017)
- [Stop Misusing t-SNE and UMAP for Visual Analytics](https://mlsanity.com/paper/2506.08725/) (2025)
- [Low-dimensional embeddings of high-dimensional data](https://mlsanity.com/paper/2508.15929/) (2025)
- [HyperNP: Interactive Visual Exploration of Multidimensional Projection Hyperparameters](https://mlsanity.com/paper/2106.13777/) (2021)
- [Assessing and improving reliability of neighbor embedding methods: a map-continuity perspective](https://mlsanity.com/paper/2410.16608/) (2024)
- [Are machine learning interpretations reliable? A stability study on global interpretations](https://mlsanity.com/paper/2505.15728/) (2025)
- [Unsupervised Machine Learning for Scientific Discovery: Workflow and Best Practices](https://mlsanity.com/paper/2506.04553/) (2025)
- [Efficient and Robust Bayesian Selection of Hyperparameters in Dimension Reduction for Visualization](https://mlsanity.com/paper/2306.00357/) (2023)
- [Automated Manifold Learning for Reduced Order Modeling](https://mlsanity.com/paper/2506.01741/) (2025)
- [Consensus dimension reduction via multi-view learning](https://mlsanity.com/paper/2512.15802/) (2025)

## Abstract (arXiv)

Low-dimensional embeddings are widely used as visual summaries of high-dimensional data and to enable downstream scientific discoveries. Yet, popular nonlinear dimension reduction methods, such as t-SNE and UMAP, are often selected based on visual appeal alone and without rigorous quantitative validation. A major reason is that manifold embeddings typically do not provide an out-of-sample map nor an inverse back to the original feature space; this makes held-out validation, the gold standard in supervised learning, all but impossible. To address these challenges, we develop a novel framework, MEDAL (Manifold Embedding Distillation via Autoencoder Learning), which distills a fitted manifold embedding into a reusable encoder--decoder model. MEDAL trains a constrained autoencoder whose bottleneck exactly matches any teacher embedding while the decoder reconstructs the original input; this yields an explicit map for new samples, an approximate inverse, and a pointwise reconstruction-based measure of distortion in the manifold space. This converts static manifold embeddings into models that can be evaluated on held-out data, enabling quantitative validation including comparing different dimension reduction methods as well as hyperparameter tuning. Across multiple benchmark and scientific case studies, we show that MEDAL enables held-out validation to determine optimal manifold embeddings and hyperparameters, reveals biologically coherent regions that are difficult to preserve in two dimensional embeddings, and detects distribution shift when new samples are mapped into a fixed reference manifold. MEDAL provides a general validation wrapper to any existing dimension reduction technique that will improve the rigor and reliability of dimension reduction in scientific workflows.