# Multi-rules mining algorithm for combinatorially exploded decision trees with modified Aitchison-Aitken function-based Bayesian optimization

arXiv: https://arxiv.org/abs/2310.02633 · HTML: https://mlsanity.com/paper/2310.02633/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 59 references that are in the map.
- First posted on arXiv 4 Oct 2023.

## Builds on (most cited first, 6 of 6)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performance](https://mlsanity.com/paper/2304.11127/) (2023)
- [Explainable Matrix -- Visualization for Global and Local Interpretability of Random Forest Classification Ensembles](https://mlsanity.com/paper/2005.04289/) (2020)
- [Unbiased variable importance for random forests](https://mlsanity.com/paper/2003.02106/) (2020)
- [Classification of Titanic Passenger Data and Chances of Surviving the Disaster](https://mlsanity.com/paper/1810.09851/) (2018)
- [Using Decision Trees for Interpretable Supervised Clustering](https://mlsanity.com/paper/2307.08104/) (2023)

## Abstract (arXiv)

Decision trees offer the benefit of easy interpretation because they allow the classification of input data based on if--then rules. However, as decision trees are constructed by an algorithm that achieves clear classification with minimum necessary rules, the trees possess the drawback of extracting only minimum rules, even when various latent rules exist in data. Approaches that construct multiple trees using randomly selected feature subsets do exist. However, the number of trees that can be constructed remains at the same scale because the number of feature subsets is a combinatorial explosion. Additionally, when multiple trees are constructed, numerous rules are generated, of which several are untrustworthy and/or highly similar. Therefore, we propose "MAABO-MT" and "GS-MRM" algorithms that strategically construct trees with high estimation performance among all possible trees with small computational complexity and extract only reliable and non-similar rules, respectively. Experiments are conducted using several open datasets to analyze the effectiveness of the proposed method. The results confirm that MAABO-MT can discover reliable rules at a lower computational cost than other methods that rely on randomness. Furthermore, the proposed method is confirmed to provide deeper insights than single decision trees commonly used in previous studies. Therefore, MAABO-MT and GS-MRM can efficiently extract rules from combinatorially exploded decision trees.