# Conditional Density Estimation with Histogram Trees

arXiv: https://arxiv.org/abs/2410.11449 · HTML: https://mlsanity.com/paper/2410.11449/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 61 references that are in the map.
- First posted on arXiv 15 Oct 2024.

## Builds on (most cited first, 19 of 19)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Optimal Sparse Decision Trees](https://mlsanity.com/paper/1904.12847/) (2019)
- [Conditional Density Estimation with Bayesian Normalising Flows](https://mlsanity.com/paper/1802.04908/) (2018)
- [Conditional Density Estimation with Neural Networks: Best Practices and Benchmarks](https://mlsanity.com/paper/1903.00954/) (2019)
- [Minimum Description Length Revisited](https://mlsanity.com/paper/1908.08484/) (2019)
- [Gaussian Process Conditional Density Estimation](https://mlsanity.com/paper/1810.12750/) (2018)
- [Noise Regularization for Conditional Density Estimation](https://mlsanity.com/paper/1907.08982/) (2019)
- [Interpretable multiclass classification by MDL-based rule lists](https://mlsanity.com/paper/1905.00328/) (2019)
- [The Minimum Description Length Principle for Pattern Mining: A Survey](https://mlsanity.com/paper/2007.14009/) (2020)
- [LinCDE: Conditional Density Estimation via Lindsey's Method](https://mlsanity.com/paper/2107.12713/) (2021)
- [Optimal Sparse Regression Trees](https://mlsanity.com/paper/2211.14980/) (2022)
- [RFCDE: Random Forests for Conditional Density Estimation](https://mlsanity.com/paper/1804.05753/) (2018)
- [Posterior Representations for Bayesian Context Trees: Sampling, Estimation and Convergence](https://mlsanity.com/paper/2202.02239/) (2022)
- [Truly Unordered Probabilistic Rule Sets for Multi-class Classification](https://mlsanity.com/paper/2206.08804/) (2022)
- [Bottleneck Conditional Density Estimation](https://mlsanity.com/paper/1611.08568/) (2016)
- [Dirac Delta Regression: Conditional Density Estimation with Clinical Trials](https://mlsanity.com/paper/1905.10330/) (2019)
- [Unsupervised Discretization by Two-dimensional MDL-based Histogram](https://mlsanity.com/paper/2006.01893/) (2020)
- [Quotient Normalized Maximum Likelihood Criterion for Learning Bayesian Network Structures](https://mlsanity.com/paper/2408.14935/) (2024)

## Built on by (most cited first, 4 of 4)

- [Nonparametric estimation of conditional probability distributions using a generative approach based on conditional push-forward neural networks](https://mlsanity.com/paper/2511.14455/) (2025)
- [Learning Subgroups with Maximum Treatment Effects without Causal Heuristics](https://mlsanity.com/paper/2511.20189/) (2025)
- [Partition Tree: Conditional Density Estimation over General Outcome Spaces](https://mlsanity.com/paper/2602.04042/) (2026)
- [XMatch: Enhancing Covariate-Aware Time Series Forecasting through Tree-Structured Exogenous Matching](https://mlsanity.com/paper/2609.34939/) (2026)

## Abstract (arXiv)

Conditional density estimation (CDE) goes beyond regression by modeling the full conditional distribution, providing a richer understanding of the data than just the conditional mean in regression. This makes CDE particularly useful in critical application domains. However, interpretable CDE methods are understudied. Current methods typically employ kernel-based approaches, using kernel functions directly for kernel density estimation or as basis functions in linear models. In contrast, despite their conceptual simplicity and visualization suitability, tree-based methods -- which are arguably more comprehensible -- have been largely overlooked for CDE tasks. Thus, we propose the Conditional Density Tree (CDTree), a fully non-parametric model consisting of a decision tree in which each leaf is formed by a histogram model. Specifically, we formalize the problem of learning a CDTree using the minimum description length (MDL) principle, which eliminates the need for tuning the hyperparameter for regularization. Next, we propose an iterative algorithm that, although greedily, searches the optimal histogram for every possible node split. Our experiments demonstrate that, in comparison to existing interpretable CDE methods, CDTrees are both more accurate (as measured by the log-loss) and more robust against irrelevant features. Further, our approach leads to smaller tree sizes than existing tree-based models, which benefits interpretability.