# Sparse Oblique Rule Boosting for Simpler Additive Rule Ensembles

arXiv: https://arxiv.org/abs/2609.06426 · HTML: https://mlsanity.com/paper/2609.06426/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 37 references that are in the map.
- First posted on arXiv 6 Sep 2026.

## Builds on (most cited first, 10 of 10)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Interpretable machine learning: definitions, methods, and applications](https://mlsanity.com/paper/1901.04592/) (2019)
- [InterpretML: A Unified Framework for Machine Learning Interpretability](https://mlsanity.com/paper/1909.09223/) (2019)
- [Learning Optimized Risk Scores](https://mlsanity.com/paper/1610.00168/) (2016)
- [Learning Accurate and Interpretable Decision Rule Sets from Neural Networks](https://mlsanity.com/paper/2103.02826/) (2021)
- [Interpretable Random Forests via Rule Extraction](https://mlsanity.com/paper/2004.14841/) (2020)
- [Learning Interpretable Rules for Scalable Data Representation and Classification](https://mlsanity.com/paper/2310.14336/) (2023)
- [Better Short than Greedy: Interpretable Models through Optimal Rule Boosting](https://mlsanity.com/paper/2101.08380/) (2021)
- [Neural Rule Ensembles: Encoding Sparse Feature Interactions into Neural Networks](https://mlsanity.com/paper/2002.04319/) (2020)
- [Orthogonal Gradient Boosting for Simpler Additive Rule Ensembles](https://mlsanity.com/paper/2402.15691/) (2024)

## Abstract (arXiv)

Small additive ensembles of symbolic rules offer interpretable prediction models. Traditionally, these ensembles use rule conditions based on conjunctions of simple threshold propositions x ≥ t on a single input variable x and threshold t, resulting geometrically in axis-parallel polytopes as decision regions. While this form ensures a high degree of interpretability for individual rules and can be learned efficiently using the gradient boosting approach, it relies on having access to a curated set of expressive input features so that a small ensemble of axis-parallel regions can describe the target variable well. Absent such features, reaching sufficient accuracy requires increasing the number and complexity of individual rules, which diminishes the interpretability of the model. Here, we extend classical rule ensembles by introducing logical propositions with learnable sparse linear transformations of input variables, i.e., propositions of the form xTw ≥ t, where w is a learnable sparse weight vector, enabling decision regions as general polyhedrons with oblique faces. We propose a learning method using gradient boosting based on a weighted logistic regression. Empirical results across 14 regression and classification tasks demonstrate that the proposed method achieves lower model complexity than competitive baselines while maintaining similar or better predictive accuracy. Hence, the approach provides a favorable trade-off between interpretability and accuracy and reduces the reliance on manual feature engineering.