# Towards Stable Machine Learning Model Retraining via Slowly Varying Sequences

arXiv: https://arxiv.org/abs/2403.19871 · HTML: https://mlsanity.com/paper/2403.19871/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 52 references that are in the map.
- First posted on arXiv 28 Mar 2024.

## Builds on (most cited first, 22 of 22)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Interpretable machine learning: definitions, methods, and applications](https://mlsanity.com/paper/1901.04592/) (2019)
- [Online Learning: A Comprehensive Survey](https://mlsanity.com/paper/1802.02871/) (2018)
- [DeltaGrad: Rapid retraining of machine learning models](https://mlsanity.com/paper/2006.14755/) (2020)
- [Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time](https://mlsanity.com/paper/2211.14238/) (2022)
- [Repairing without Retraining: Avoiding Disparate Impact with Counterfactual Distributions](https://mlsanity.com/paper/1901.10501/) (2019)
- [Iterative Random Forests to detect predictive and stable high-order interactions](https://mlsanity.com/paper/1706.08457/) (2017)
- [How to Retrain Recommender System? A Sequential Meta-Learning Method](https://mlsanity.com/paper/2005.13258/) (2020)
- [Continual Learning in Practice](https://mlsanity.com/paper/1903.05202/) (2019)
- [Improving Robustness against Real-World and Worst-Case Distribution Shifts through Decision Region Quantification](https://mlsanity.com/paper/2205.09619/) (2022)
- [Causal Rule Ensemble: Interpretable Discovery and Inference of Heterogeneous Treatment Effects](https://mlsanity.com/paper/2009.09036/) (2020)
- [Adaptation Strategies for Automated Machine Learning on Evolving Data](https://mlsanity.com/paper/2006.06480/) (2020)
- [Slowly Varying Regression under Sparsity](https://mlsanity.com/paper/2102.10773/) (2021)
- [Cost-Effective Retraining of Machine Learning Models](https://mlsanity.com/paper/2310.04216/) (2023)
- [Algorithmic Insurance](https://mlsanity.com/paper/2106.00839/) (2021)
- [Generalization In Multi-Objective Machine Learning](https://mlsanity.com/paper/2208.13499/) (2022)
- [Stability of Random Forests and Coverage of Random-Forest Prediction Intervals](https://mlsanity.com/paper/2310.18814/) (2023)
- [Patient Outcome Predictions Improve Operations at a Large Hospital Network](https://mlsanity.com/paper/2305.15629/) (2023)
- [Improving Stability in Decision Tree Models](https://mlsanity.com/paper/2305.17299/) (2023)

## Built on by (most cited first, 2 of 2)

- [The Challenger: When Do New Data Sources Justify Switching Machine Learning Models?](https://mlsanity.com/paper/2512.18390/) (2025)
- [An empirical evaluation of the risks of AI model updates using clinical data: stability, arbitrariness, and fairness](https://mlsanity.com/paper/2604.23954/) (2026)

## Abstract (arXiv)

We consider the problem of retraining machine learning (ML) models when new batches of data become available. Existing approaches greedily optimize for predictive power independently at each batch, without considering the stability of the model's structure or analytical insights across retraining iterations. We propose a model-agnostic framework for finding sequences of models that are stable across retraining iterations. We develop a mixed-integer optimization formulation that is guaranteed to recover Pareto optimal models (in terms of the predictive power-stability trade-off) with good generalization properties, as well as an efficient polynomial-time algorithm that performs well in practice. We focus on retaining consistent analytical insights-which is important to model interpretability, ease of implementation, and fostering trust with users-by using custom-defined distance metrics that can be directly incorporated into the optimization problem. We evaluate our framework across models (regression, decision trees, boosted trees, and neural networks) and application domains (healthcare, vision, and language), including deployment in a production pipeline at a major US hospital. We find that, on average, a 2% reduction in predictive power leads to a 30% improvement in stability.