# Autoencoding Random Forests

arXiv: https://arxiv.org/abs/2505.21441 · HTML: https://mlsanity.com/paper/2505.21441/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 115 references that are in the map.
- First posted on arXiv 27 May 2025.

## Builds on (most cited first, 29 of 29)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Tabular Data: Deep Learning is Not All You Need](https://mlsanity.com/paper/2106.03253/) (2021)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Generalized Random Forests](https://mlsanity.com/paper/1610.01271/) (2016)
- [On Completeness-aware Concept-Based Explanations in Deep Neural Networks](https://mlsanity.com/paper/1910.07969/) (2019)
- [Distilling a Neural Network Into a Soft Decision Tree](https://mlsanity.com/paper/1711.09784/) (2017)
- [Language Modeling Is Compression](https://mlsanity.com/paper/2309.10668/) (2023)
- [Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Space](https://mlsanity.com/paper/2310.09656/) (2023)
- [ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and R](https://mlsanity.com/paper/1508.04409/) (2015)
- [A Random Forest Guided Tour](https://mlsanity.com/paper/1511.05741/) (2015)
- [Exploration in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2205.00824/) (2022)
- [Adversarial random forests for density estimation and generative modeling](https://mlsanity.com/paper/2205.09435/) (2022)
- [Mondrian Forests: Efficient Online Random Forests](https://mlsanity.com/paper/1406.2673/) (2014)
- [Joints in Random Forests](https://mlsanity.com/paper/2006.14937/) (2020)
- [The Random Forest Kernel and other kernels for big data from random partitions](https://mlsanity.com/paper/1402.4293/) (2014)
- [Minimax optimal rates for Mondrian trees and forests](https://mlsanity.com/paper/1803.05784/) (2018)
- [Narrowing the Gap: Random Forests In Theory and In Practice](https://mlsanity.com/paper/1310.1415/) (2013)
- [The Mondrian Kernel](https://mlsanity.com/paper/1606.05241/) (2016)
- [AutoEncoder by Forest](https://mlsanity.com/paper/1709.09018/) (2017)
- [Boulevard: Regularized Stochastic Gradient Boosted Trees and Their Limiting Distribution](https://mlsanity.com/paper/1806.09762/) (2018)
- [Inference with Mondrian Random Forests](https://mlsanity.com/paper/2310.09702/) (2023)
- [Within Group Variable Selection through the Exclusive Lasso](https://mlsanity.com/paper/1505.07517/) (2015)
- [Random Forest Autoencoders for Guided Representation Learning](https://mlsanity.com/paper/2502.13257/) (2025)
- [Learning Interpretable Characteristic Kernels via Decision Forests](https://mlsanity.com/paper/1812.00029/) (2018)
- [Kernel PCA with the Nystr"om method](https://mlsanity.com/paper/2109.05578/) (2021)
- [A Highly Efficient Algorithm for Solving Exclusive Lasso Problems](https://mlsanity.com/paper/2306.14196/) (2023)

## Built on by (most cited first, 3 of 3)

- [Revisiting Forest Proximities via Sparse Leaf-Incidence Kernels](https://mlsanity.com/paper/2601.02735/) (2026)
- [Minimax Rates and Spectral Distillation for Tree Ensembles](https://mlsanity.com/paper/2605.11841/) (2026)
- [Kernel of Partition Paths: A Unified Representation for Tree Ensembles](https://mlsanity.com/paper/2606.18853/) (2026)

## Abstract (arXiv)

We propose a principled method for autoencoding with random forests. Our strategy builds on foundational results from nonparametric statistics and spectral graph theory to learn a low-dimensional embedding of the model that optimally represents relationships in the data. We provide exact and approximate solutions to the decoding problem via constrained optimization, split relabeling, and nearest neighbors regression. These methods effectively invert the compression pipeline, establishing a map from the embedding space back to the input space using splits learned by the ensemble's constituent trees. The resulting decoders are universally consistent under common regularity assumptions. The procedure works with supervised or unsupervised models, providing a window into conditional or joint distributions. We demonstrate various applications of this autoencoder, including powerful new tools for visualization, compression, clustering, and denoising. Experiments illustrate the ease and utility of our method in a wide range of settings, including tabular, image, and genomic data.