# Stabilizing black-box model selection with the inflated argmax

arXiv: https://arxiv.org/abs/2410.18268 · HTML: https://mlsanity.com/paper/2410.18268/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 45 references that are in the map.
- First posted on arXiv 23 Oct 2024.

## Builds on (most cited first, 12 of 12)

- [Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning](https://mlsanity.com/paper/1811.12808/) (2018)
- [Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control](https://mlsanity.com/paper/2111.10992/) (2021)
- [Stability](https://mlsanity.com/paper/1310.0150/) (2013)
- [PySINDy: A comprehensive Python package for robust sparse system identification](https://mlsanity.com/paper/2111.08481/) (2021)
- [Exploring the Whole Rashomon Set of Sparse Decision Trees](https://mlsanity.com/paper/2209.08040/) (2022)
- [Machine Learning Advances for Time Series Forecasting](https://mlsanity.com/paper/2012.12802/) (2020)
- [Model Selection Techniques -- An Overview](https://mlsanity.com/paper/1810.09583/) (2018)
- [Bagging Provides Assumption-free Stability](https://mlsanity.com/paper/2301.12600/) (2023)
- [Building a stable classifier with the inflated argmax](https://mlsanity.com/paper/2405.14064/) (2024)
- [Winners with Confidence: Discrete Argmin Inference with an Application to Model Selection](https://mlsanity.com/paper/2408.02060/) (2024)
- [Reproducible Model Selection Using Bagged Posteriors](https://mlsanity.com/paper/2007.14845/) (2020)
- [Discussion of Big Bayes Stories and BayesBag](https://mlsanity.com/paper/1405.4977/) (2014)

## Built on by (most cited first, 3 of 3)

- [Ill-Conditioning in Dictionary-Based Dynamic-Equation Learning: A Systems Biology Case Study](https://mlsanity.com/paper/2603.11330/) (2026)
- [Feature Bagging Provides Stability](https://mlsanity.com/paper/2607.26964/) (2026)
- [When May a Model Replace the Experiment? Audits, Licenses, and the Price of Trust in Surrogate-Driven Design](https://mlsanity.com/paper/2608.01378/) (2026)

## Abstract (arXiv)

Model selection is the process of choosing from a class of candidate models given data. For instance, methods such as the LASSO and sparse identification of nonlinear dynamics (SINDy) formulate model selection as finding a sparse solution to a linear system of equations determined by training data. However, absent strong assumptions, such methods are highly unstable: if a single data point is removed from the training set, a different model may be selected. In this paper, we present a new approach to stabilizing model selection with theoretical stability guarantees that leverages a combination of bagging and an ''inflated'' argmax operation. Our method selects a small collection of models that all fit the data, and it is stable in that, with high probability, the removal of any training point will result in a collection of selected models that overlaps with the original collection. We illustrate this method in (a) a simulation in which strongly correlated covariates make standard LASSO model selection highly unstable, (b) a Lotka-Volterra model selection problem focused on identifying how competition in an ecosystem influences species' abundances, (c) a graph subset selection problem using cell-signaling data from proteomics, and (d) unsupervised kappa-means clustering. In these settings, the proposed method yields stable, compact, and accurate collections of selected models, outperforming a variety of benchmarks.