# SplitWise Regression: Stepwise Modeling with Adaptive Dummy Encoding

arXiv: https://arxiv.org/abs/2505.15423 · HTML: https://mlsanity.com/paper/2505.15423/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 55 references that are in the map.
- First posted on arXiv 21 May 2025.

## Builds on (most cited first, 9 of 9)

- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [Bayesian Additive Regression Trees with Model Trees](https://mlsanity.com/paper/2006.07493/) (2020)
- [Challenging the Performance-Interpretability Trade-off: An Evaluation of Interpretable Machine Learning Models](https://mlsanity.com/paper/2409.14429/) (2024)
- [Fast Linear Model Trees by PILOT](https://mlsanity.com/paper/2302.03931/) (2023)
- [Fitting Prediction Rule Ensembles with R Package pre](https://mlsanity.com/paper/1707.07149/) (2017)
- [BiometricBlender: Ultra-high dimensional, multi-class synthetic data generator to imitate biometric feature space](https://mlsanity.com/paper/2206.10747/) (2022)
- [Feature space reduction method for ultrahigh-dimensional, multiclass data: Random forest-based multiround screening (RFMS)](https://mlsanity.com/paper/2305.15793/) (2023)

## Abstract (arXiv)

Capturing nonlinear relationships without sacrificing interpretability remains a persistent challenge in regression modeling. We introduce SplitWise, a novel framework that enhances stepwise regression. It adaptively transforms numeric predictors into threshold-based binary features using shallow decision trees, but only when such transformations improve model fit, as assessed by the Akaike Information Criterion (AIC) or Bayesian Information Criterion (BIC). This approach preserves the transparency of linear models while flexibly capturing nonlinear effects. Implemented as a user-friendly R package, SplitWise is evaluated on both synthetic and real-world datasets. The results show that it consistently produces more parsimonious and generalizable models than traditional stepwise and penalized regression techniques.