# Joints in Random Forests

arXiv: https://arxiv.org/abs/2006.14937 · HTML: https://mlsanity.com/paper/2006.14937/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jun 2020.

## Built on by (most cited first, 21 of 21)

- [Semantic Probabilistic Layers for Neuro-Symbolic Learning](https://mlsanity.com/paper/2206.00426/) (2022)
- [Tractable Control for Autoregressive Language Generation](https://mlsanity.com/paper/2304.07438/) (2023)
- [Adversarial random forests for density estimation and generative modeling](https://mlsanity.com/paper/2205.09435/) (2022)
- [Scaling Tractable Probabilistic Circuits: A Systems Perspective](https://mlsanity.com/paper/2406.00766/) (2024)
- [Continuous Mixtures of Tractable Probabilistic Models](https://mlsanity.com/paper/2209.10584/) (2022)
- [Understanding the Distillation Process from Deep Generative Models to Tractable Probabilistic Circuits](https://mlsanity.com/paper/2302.08086/) (2023)
- [Sparse Probabilistic Circuits via Pruning and Growing](https://mlsanity.com/paper/2211.12551/) (2022)
- [Scaling Continuous Latent Variable Models as Probabilistic Integral Circuits](https://mlsanity.com/paper/2406.06494/) (2024)
- [A Compositional Atlas of Tractable Circuit Operations: From Simple Transformations to Complex Information-Theoretic Queries](https://mlsanity.com/paper/2102.06137/) (2021)
- [Autoencoding Random Forests](https://mlsanity.com/paper/2505.21441/) (2025)
- [Undersmoothing Causal Estimators with Generative Trees](https://mlsanity.com/paper/2203.08570/) (2022)
- [Probabilistic Circuits with Constraints via Convex Optimization](https://mlsanity.com/paper/2403.13125/) (2024)
- [Soft Learning Probabilistic Circuits](https://mlsanity.com/paper/2403.14504/) (2024)
- [NRGBoost: Energy-Based Generative Boosted Trees](https://mlsanity.com/paper/2410.03535/) (2024)
- [Robustness Quantification and Uncertainty Quantification: Comparing Two Methods for Assessing the Reliability of Classifier Predictions](https://mlsanity.com/paper/2603.22988/) (2026)
- [A Sobering Look at Tabular Data Generation via Probabilistic Circuits](https://mlsanity.com/paper/2603.23016/) (2026)
- [Sampling - Variational Auto Encoder - Ensemble: In the Quest of Explainable Artificial Intelligence](https://mlsanity.com/paper/2309.14385/) (2023)
- [Probably Approximately Correct Maximum A Posteriori Inference](https://mlsanity.com/paper/2601.16083/) (2026)
- [Machine Learning in Epidemiology](https://mlsanity.com/paper/2602.16352/) (2026)
- [A Probabilistic Circuit-Induced Pseudo-Metric for Out-of-Distribution Detection](https://mlsanity.com/paper/2608.09117/) (2026)
- [Local Robustness Quantification for Naive Bayes Classifiers and Generative Forests: a General Approach](https://mlsanity.com/paper/2609.11366/) (2026)

## Abstract (arXiv)

Decision Trees (DTs) and Random Forests (RFs) are powerful discriminative learners and tools of central importance to the everyday machine learning practitioner and data scientist. Due to their discriminative nature, however, they lack principled methods to process inputs with missing features or to detect outliers, which requires pairing them with imputation techniques or a separate generative model. In this paper, we demonstrate that DTs and RFs can naturally be interpreted as generative models, by drawing a connection to Probabilistic Circuits, a prominent class of tractable probabilistic models. This reinterpretation equips them with a full joint distribution over the feature space and leads to Generative Decision Trees (GeDTs) and Generative Forests (GeFs), a family of novel hybrid generative-discriminative models. This family of models retains the overall characteristics of DTs and RFs while additionally being able to handle missing features by means of marginalisation. Under certain assumptions, frequently made for Bayes consistency results, we show that consistency in GeDTs and GeFs extend to any pattern of missing input features, if missing at random. Empirically, we show that our models often outperform common routines to treat missing data, such as K-nearest neighbour imputation, and moreover, that our models can naturally detect outliers by monitoring the marginal probability of input features.