# Interpretable Ensembles of Hyper-Rectangles as Base Models

arXiv: https://arxiv.org/abs/2303.08625 · HTML: https://mlsanity.com/paper/2303.08625/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 94 references that are in the map.
- First posted on arXiv 15 Mar 2023.

## Builds on (most cited first, 30 of 36)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Consistent Individualized Feature Attribution for Tree Ensembles](https://mlsanity.com/paper/1802.03888/) (2018)
- [On Explainability of Graph Neural Networks via Subgraph Explorations](https://mlsanity.com/paper/2102.05152/) (2021)
- [Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey](https://mlsanity.com/paper/2006.11371/) (2020)
- [Explaining by Removing: A Unified Framework for Model Explanation](https://mlsanity.com/paper/2011.14878/) (2020)
- [GraphLIME: Local Interpretable Model Explanations for Graph Neural Networks](https://mlsanity.com/paper/2001.06216/) (2020)
- [CatBoost: gradient boosting with categorical features support](https://mlsanity.com/paper/1810.11363/) (2018)
- [Explaining individual predictions when features are dependent: More accurate approximations to Shapley values](https://mlsanity.com/paper/1903.10464/) (2019)
- [Explainable Deep Learning: A Field Guide for the Uninitiated](https://mlsanity.com/paper/2004.14545/) (2020)
- [FastSHAP: Real-Time Shapley Value Estimation](https://mlsanity.com/paper/2107.07436/) (2021)
- [Principles and Practice of Explainable Machine Learning](https://mlsanity.com/paper/2009.11698/) (2020)
- [On the Tractability of SHAP Explanations](https://mlsanity.com/paper/2009.08634/) (2020)
- [A Comparative Analysis of XGBoost](https://mlsanity.com/paper/1911.01914/) (2019)
- [TimeSHAP: Explaining Recurrent Models through Sequence Perturbations](https://mlsanity.com/paper/2012.00073/) (2020)
- [SurvLIME: A method for explaining machine learning survival models](https://mlsanity.com/paper/2003.08371/) (2020)
- [Explainability for fair machine learning](https://mlsanity.com/paper/2010.07389/) (2020)
- [Explaining Anomalies Detected by Autoencoders Using SHAP](https://mlsanity.com/paper/1903.02407/) (2019)
- [Enriching Visual with Verbal Explanations for Relational Concepts -- Combining LIME with Aleph](https://mlsanity.com/paper/1910.01837/) (2019)
- [The Shapley Value of Classifiers in Ensemble Games](https://mlsanity.com/paper/2101.02153/) (2021)
- [SHAFF: Fast and consistent SHApley eFfect estimates via random Forests](https://mlsanity.com/paper/2105.11724/) (2021)
- [Looking Deeper into Tabular LIME](https://mlsanity.com/paper/2008.11092/) (2020)
- [Shapley Values of Reconstruction Errors of PCA for Explaining Anomaly Detection](https://mlsanity.com/paper/1909.03495/) (2019)
- [The Evolution of Boosting Algorithms - From Machine Learning to Statistical Modelling](https://mlsanity.com/paper/1403.1452/) (2014)
- [Interpretable Machine Learning with an Ensemble of Gradient Boosting Machines](https://mlsanity.com/paper/2010.07388/) (2020)
- [Accurate Shapley Values for explaining tree-based models](https://mlsanity.com/paper/2106.03820/) (2021)
- [Ensembles of Random SHAPs](https://mlsanity.com/paper/2103.03302/) (2021)

## Built on by (most cited first, 2 of 2)

- [A novel gradient-based method for decision trees optimizing arbitrary differential loss functions](https://mlsanity.com/paper/2503.17855/) (2025)
- [Differentiable Interval Bottlenecks for Interpretable Anomaly Detection in Numerical Data](https://mlsanity.com/paper/2609.03878/) (2026)

## Abstract (arXiv)

A new extremely simple ensemble-based model with the uniformly generated axis-parallel hyper-rectangles as base models (HRBM) is proposed. Two types of HRBMs are studied: closed rectangles and corners. The main idea behind HRBM is to consider and count training examples inside and outside each rectangle. It is proposed to incorporate HRBMs into the gradient boosting machine (GBM). Despite simplicity of HRBMs, it turns out that these simple base models allow us to construct effective ensemble-based models and avoid overfitting. A simple method for calculating optimal regularization parameters of the ensemble-based model, which can be modified in the explicit way at each iteration of GBM, is considered. Moreover, a new regularization called the "step height penalty" is studied in addition to the standard L1 and L2 regularizations. An extremely simple approach to the proposed ensemble-based model prediction interpretation by using the well-known method SHAP is proposed. It is shown that GBM with HRBM can be regarded as a model extending a set of interpretable models for explaining black-box models. Numerical experiments with real datasets illustrate the proposed GBM with HRBMs for regression and classification problems. Experiments also illustrate computational efficiency of the proposed SHAP modifications. The code of proposed algorithms implementing GBM with HRBM is publicly available.