# Forward Stability and Model Path Selection

arXiv: https://arxiv.org/abs/2103.03462 · HTML: https://mlsanity.com/paper/2103.03462/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Mar 2021.

## Built on by (most cited first, 12 of 12)

- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [Exploring the Whole Rashomon Set of Sparse Decision Trees](https://mlsanity.com/paper/2209.08040/) (2022)
- [Exploring and Interacting with the Set of Good Sparse Generalized Additive Models](https://mlsanity.com/paper/2303.16047/) (2023)
- [Systemizing Multiplicity: The Curious Case of Arbitrariness in Machine Learning](https://mlsanity.com/paper/2501.14959/) (2025)
- [TimberTrek: Exploring and Curating Sparse Decision Trees with Interactive Visualization](https://mlsanity.com/paper/2209.09227/) (2022)
- [Partial Order in Chaos: Consensus on Feature Attributions in the Rashomon Set](https://mlsanity.com/paper/2110.13369/) (2021)
- [Exploration of the Rashomon Set Assists Trustworthy Explanations for Medical Data](https://mlsanity.com/paper/2308.11446/) (2023)
- [Bayesian subset selection and variable importance for interpretable prediction and classification](https://mlsanity.com/paper/2104.10150/) (2021)
- [X Hacking: The Threat of Misguided AutoML](https://mlsanity.com/paper/2401.08513/) (2024)
- [Stability Selection via Variable Decorrelation](https://mlsanity.com/paper/2505.20864/) (2025)
- ["A 6 or a 9?": Ensemble Learning Through the Multiplicity of Performant Models and Explanations](https://mlsanity.com/paper/2509.09073/) (2025)
- [Data as a Lever: A Neighbouring Datasets Perspective on Predictive Multiplicity](https://mlsanity.com/paper/2510.21303/) (2025)

## Abstract (arXiv)

Most scientific publications follow the familiar recipe of (i) obtain data, (ii) fit a model, and (iii) comment on the scientific relevance of the effects of particular covariates in that model. This approach, however, ignores the fact that there may exist a multitude of similarly-accurate models in which the implied effects of individual covariates may be vastly different. This problem of finding an entire collection of plausible models has also received relatively little attention in the statistics community, with nearly all of the proposed methodologies being narrowly tailored to a particular model class and/or requiring an exhaustive search over all possible models, making them largely infeasible in the current big data era. This work develops the idea of forward stability and proposes a novel, computationally-efficient approach to finding collections of accurate models we refer to as model path selection (MPS). MPS builds up a plausible model collection via a forward selection approach and is entirely agnostic to the model class and loss function employed. The resulting model collection can be displayed in a simple and intuitive graphical fashion, easily allowing practitioners to visualize whether some covariates can be swapped for others with minimal loss.