# Random Forest Autoencoders for Guided Representation Learning

arXiv: https://arxiv.org/abs/2502.13257 · HTML: https://mlsanity.com/paper/2502.13257/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 89 references that are in the map.
- First posted on arXiv 18 Feb 2025.

## Builds on (most cited first, 26 of 26)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification](https://mlsanity.com/paper/2110.14795/) (2021)
- [Lossy Image Compression with Compressive Autoencoders](https://mlsanity.com/paper/1703.00395/) (2017)
- [MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis](https://mlsanity.com/paper/2010.14925/) (2020)
- [Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization](https://mlsanity.com/paper/2012.04456/) (2020)
- [Learnable latent embeddings for joint behavioral and neural analysis](https://mlsanity.com/paper/2204.00673/) (2022)
- [Parametric UMAP embeddings for representation and semi-supervised learning](https://mlsanity.com/paper/2009.12981/) (2020)
- [Dropout Training as Adaptive Regularization](https://mlsanity.com/paper/1307.1493/) (2013)
- [Less is More: Nystr"om Computational Regularization](https://mlsanity.com/paper/1507.04717/) (2015)
- [Faster k-Medoids Clustering: Improving the PAM, CLARA, and CLARANS Algorithms](https://mlsanity.com/paper/1810.05691/) (2018)
- [Fast and Eager k-Medoids Clustering: O(k) Runtime Improvement of the PAM, CLARA, and CLARANS Algorithms](https://mlsanity.com/paper/2008.05171/) (2020)
- [From t-SNE to UMAP with contrastive learning](https://mlsanity.com/paper/2206.01816/) (2022)
- [Learning Internal Representations (COLT 1995)](https://mlsanity.com/paper/1911.05781/) (2019)
- [Geometry- and Accuracy-Preserving Random Forest Proximities](https://mlsanity.com/paper/2201.12682/) (2022)
- [Geometric Autoencoders -- What You See is What You Decode](https://mlsanity.com/paper/2306.17638/) (2023)
- [Heavy-tailed kernels reveal a finer cluster structure in t-SNE visualisations](https://mlsanity.com/paper/1902.05804/) (2019)
- [Extendable and invertible manifold learning with geometry regularized autoencoders](https://mlsanity.com/paper/2007.07142/) (2020)
- [Supervised Dimensionality Reduction for Big Data](https://mlsanity.com/paper/1709.01233/) (2017)
- [Supervised Dimensionality Reduction and Visualization using Centroid-encoder](https://mlsanity.com/paper/2002.11934/) (2020)
- [Stochastic Neighbor Embedding under f-divergences](https://mlsanity.com/paper/1811.01247/) (2018)
- [Compressed Diffusion](https://mlsanity.com/paper/1902.00033/) (2019)
- [Sparse Projections of Medical Images onto Manifolds](https://mlsanity.com/paper/1303.5508/) (2013)
- [Enhancing Supervised Visualization through Autoencoder and Random Forest Proximities for Out-of-Sample Extension](https://mlsanity.com/paper/2406.04421/) (2024)

## Built on by (most cited first, 4 of 4)

- [Autoencoding Random Forests](https://mlsanity.com/paper/2505.21441/) (2025)
- [Revisiting Forest Proximities via Sparse Leaf-Incidence Kernels](https://mlsanity.com/paper/2601.02735/) (2026)
- [Freeze, Diffuse, Decode: Task-Aware Adaptation of Transformer Embeddings for Antimicrobial Peptide Design](https://mlsanity.com/paper/2511.23120/) (2025)
- [Forest-Guided Semantic Transport for Label-Supervised Manifold Alignment](https://mlsanity.com/paper/2602.00974/) (2026)

## Abstract (arXiv)

Extensive research has produced robust methods for unsupervised data visualization. Yet supervised visualizationunicodex2013where expert labels guide representationsunicodex2013remains underexplored, as most supervised approaches prioritize classification over visualization. Recently, RF-PHATE, a diffusion-based manifold learning method leveraging random forests and information geometry, marked significant progress in supervised visualization. However, its lack of an explicit mapping function limits scalability and its application to unseen data, posing challenges for large datasets and label-scarce scenarios. To overcome these limitations, we introduce Random Forest Autoencoders (RF-AE), a neural network-based framework for out-of-sample kernel extension that combines the flexibility of autoencoders with the supervised learning strengths of random forests and the geometry captured by RF-PHATE. RF-AE enables efficient out-of-sample supervised visualization and outperforms existing methods, including RF-PHATE's standard kernel extension, in both accuracy and interpretability. Additionally, RF-AE is robust to the choice of hyperparameters and generalizes to any kernel-based dimensionality reduction method.