# Born-Again Tree Ensembles

arXiv: https://arxiv.org/abs/2003.11132 · HTML: https://mlsanity.com/paper/2003.11132/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Mar 2020.

## Built on by (most cited first, 30 of 38)

- [Fast Sparse Decision Tree Optimization via Reference Ensembles](https://mlsanity.com/paper/2112.00798/) (2021)
- [Optimal Counterfactual Explanations in Tree Ensembles](https://mlsanity.com/paper/2106.06631/) (2021)
- [Provably efficient, succinct, and precise explanations](https://mlsanity.com/paper/2111.01576/) (2021)
- [On the price of explainability for some clustering problems](https://mlsanity.com/paper/2101.01576/) (2021)
- [Optimal Decision Diagrams for Classification](https://mlsanity.com/paper/2205.14500/) (2022)
- [FS-BAN: Born-Again Networks for Domain Generalization Few-Shot Classification](https://mlsanity.com/paper/2208.10930/) (2022)
- [Superpolynomial Lower Bounds for Decision Tree Learning and Testing](https://mlsanity.com/paper/2210.06375/) (2022)
- [A Query-Optimal Algorithm for Finding Counterfactuals](https://mlsanity.com/paper/2207.07072/) (2022)
- [Explainable Data-Driven Optimization: From Context to Decision and Back Again](https://mlsanity.com/paper/2301.10074/) (2023)
- [A Theory of Interpretable Approximations](https://mlsanity.com/paper/2406.10529/) (2024)
- [Properly Learning Decision Trees with Queries Is NP-Hard](https://mlsanity.com/paper/2307.04093/) (2023)
- [Free Lunch in the Forest: Functionally-Identical Pruning of Boosted Tree Ensembles](https://mlsanity.com/paper/2408.16167/) (2024)
- [Leveraging Predictive Equivalence in Decision Trees](https://mlsanity.com/paper/2506.14143/) (2025)
- [Explainable Models via Compression of Tree Ensembles](https://mlsanity.com/paper/2206.07904/) (2022)
- [Conceptual Views on Tree Ensemble Classifiers](https://mlsanity.com/paper/2302.05270/) (2023)
- [On Computing Optimal Tree Ensembles](https://mlsanity.com/paper/2306.04423/) (2023)
- [From Counterfactuals to Trees: Competitive Analysis of Model Extraction Attacks](https://mlsanity.com/paper/2502.05325/) (2025)
- [Contextual Decision Trees](https://mlsanity.com/paper/2207.06355/) (2022)
- [Support Vector Machines with the Hard-Margin Loss: Optimal Training via Combinatorial Benders' Cuts](https://mlsanity.com/paper/2207.07690/) (2022)
- [Unboxing Tree Ensembles for interpretability: a hierarchical visualization tool and a multivariate optimal re-built tree](https://mlsanity.com/paper/2302.07580/) (2023)
- [Counterfactual Maps: What They Are and How to Find Them](https://mlsanity.com/paper/2602.09128/) (2026)
- [Supervised Feature Compression based on Counterfactual Analysis](https://mlsanity.com/paper/2211.09894/) (2022)
- [Simplification of Forest Classifiers and Regressors](https://mlsanity.com/paper/2212.07103/) (2022)
- [Are Logistic Models Really Interpretable?](https://mlsanity.com/paper/2406.13427/) (2024)
- [A Unified Approach to Extract Interpretable Rules from Tree Ensembles via Integer Programming](https://mlsanity.com/paper/2407.00843/) (2024)
- [RuleExplorer: A Scalable Matrix Visualization for Understanding Tree Ensemble Classifiers](https://mlsanity.com/paper/2409.03164/) (2024)
- [Cluster-Based Random Forest Visualization and Interpretation](https://mlsanity.com/paper/2507.22665/) (2025)
- [Residual-as-Teacher: Mitigating Bias Propagation in Student--Teacher Estimation](https://mlsanity.com/paper/2603.25466/) (2026)
- [PACE: Prune-And-Compress Ensemble Models](https://mlsanity.com/paper/2605.06278/) (2026)
- [Making CNNs Interpretable by Building Dynamic Sequential Decision Forests with Top-down Hierarchy Learning](https://mlsanity.com/paper/2106.02824/) (2021)

## Abstract (arXiv)

The use of machine learning algorithms in finance, medicine, and criminal justice can deeply impact human lives. As a consequence, research into interpretable machine learning has rapidly grown in an attempt to better control and fix possible sources of mistakes and biases. Tree ensembles offer a good prediction quality in various domains, but the concurrent use of multiple trees reduces the interpretability of the ensemble. Against this background, we study born-again tree ensembles, i.e., the process of constructing a single decision tree of minimum size that reproduces the exact same behavior as a given tree ensemble in its entire feature space. To find such a tree, we develop a dynamic-programming based algorithm that exploits sophisticated pruning and bounding rules to reduce the number of recursive calls. This algorithm generates optimal born-again trees for many datasets of practical interest, leading to classifiers which are typically simpler and more interpretable without any other form of compromise.