# Improving Stability in Decision Tree Models

arXiv: https://arxiv.org/abs/2305.17299 · HTML: https://mlsanity.com/paper/2305.17299/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 64 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 12 of 12)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Interpretable machine learning: definitions, methods, and applications](https://mlsanity.com/paper/1901.04592/) (2019)
- [Tackling Climate Change with Machine Learning](https://mlsanity.com/paper/1906.05433/) (2019)
- [Exploring the Whole Rashomon Set of Sparse Decision Trees](https://mlsanity.com/paper/2209.08040/) (2022)
- [Learning Optimal and Fair Decision Trees for Non-Discriminative Decision-Making](https://mlsanity.com/paper/1903.10598/) (2019)
- [Connecting Interpretability and Robustness in Decision Trees through Separation](https://mlsanity.com/paper/2102.07048/) (2021)
- [Learning Optimal Classification Trees: Strong Max-Flow Formulations](https://mlsanity.com/paper/2002.09142/) (2020)
- [On the Fairness of Machine-Assisted Human Decisions](https://mlsanity.com/paper/2110.15310/) (2021)
- [The Price of Interpretability](https://mlsanity.com/paper/1907.03419/) (2019)
- [The Backbone Method for Ultra-High Dimensional Sparse Machine Learning](https://mlsanity.com/paper/2006.06592/) (2020)
- [Slowly Varying Regression under Sparsity](https://mlsanity.com/paper/2102.10773/) (2021)
- [Algorithmic Decision-Making Safeguarded by Human Knowledge](https://mlsanity.com/paper/2211.11028/) (2022)

## Built on by (most cited first, 2 of 2)

- [Towards Stable Machine Learning Model Retraining via Slowly Varying Sequences](https://mlsanity.com/paper/2403.19871/) (2024)
- [Towards Robust Interpretable Surrogates for Optimization](https://mlsanity.com/paper/2412.01264/) (2024)

## Abstract (arXiv)

Owing to their inherently interpretable structure, decision trees are commonly used in applications where interpretability is essential. Recent work has focused on improving various aspects of decision trees, including their predictive power and robustness; however, their instability, albeit well-documented, has been addressed to a lesser extent. In this paper, we take a step towards the stabilization of decision tree models through the lens of real-world health care applications due to the relevance of stability and interpretability in this space. We introduce a new distance metric for decision trees and use it to determine a tree's level of stability. We propose a novel methodology to train stable decision trees and investigate the existence of trade-offs that are inherent to decision tree models - including between stability, predictive power, and interpretability. We demonstrate the value of the proposed methodology through an extensive quantitative and qualitative analysis of six case studies from real-world health care applications, and we show that, on average, with a small 4.6% decrease in predictive power, we gain a significant 38% improvement in the model's stability.