# Self-Healing Machine Learning: A Framework for Autonomous Adaptation in Real-World Environments

arXiv: https://arxiv.org/abs/2411.00186 · HTML: https://mlsanity.com/paper/2411.00186/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 84 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Builds on (most cited first, 19 of 19)

- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift](https://mlsanity.com/paper/1810.11953/) (2018)
- [Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning](https://mlsanity.com/paper/1811.12808/) (2018)
- [Characterizing Concept Drift](https://mlsanity.com/paper/1511.03816/) (2015)
- [Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement](https://mlsanity.com/paper/2310.08559/) (2023)
- [From Concept Drift to Model Degradation: An Overview on Performance-Aware Drift Detectors](https://mlsanity.com/paper/2203.11070/) (2022)
- [Challenges in Benchmarking Stream Learning Algorithms with Real-world Data](https://mlsanity.com/paper/2005.00113/) (2020)
- [Multicalibration for Confidence Scoring in LLMs](https://mlsanity.com/paper/2404.04689/) (2024)
- [Towards Explaining Distribution Shifts](https://mlsanity.com/paper/2210.10275/) (2022)
- [Concept Drift Detection for Streaming Data](https://mlsanity.com/paper/1504.01044/) (2015)
- [Online Boosting Adaptive Learning under Concept Drift for Multistream Classification](https://mlsanity.com/paper/2312.10841/) (2023)
- [How good is the Electricity benchmark for evaluating concept drift adaptation](https://mlsanity.com/paper/1301.3524/) (2013)
- [Concept Drift and Covariate Shift Detection Ensemble with Lagged Labels](https://mlsanity.com/paper/2012.04759/) (2020)
- [Customs Fraud Detection in the Presence of Concept Drift](https://mlsanity.com/paper/2109.14155/) (2021)
- [Automating concept-drift detection by self-evaluating predictive model degradation](https://mlsanity.com/paper/1907.08120/) (2019)

## Built on by (most cited first, 8 of 8)

- [Quantifying perturbation impacts for large language models](https://mlsanity.com/paper/2412.00868/) (2024)
- [Keeping Medical AI Healthy and Trustworthy: A Review of Detection and Correction Methods for System Degradation](https://mlsanity.com/paper/2506.17442/) (2025)
- [Tiny Autoregressive Recursive Models](https://mlsanity.com/paper/2603.08082/) (2026)
- [Deep Hierarchical Learning with Nested Subspace Networks for Large Language Models](https://mlsanity.com/paper/2509.17874/) (2025)
- [Team, Then Trim: An Assembly-Line LLM Framework for High-Quality Tabular Data Generation](https://mlsanity.com/paper/2602.04785/) (2026)
- [An empirical evaluation of the risks of AI model updates using clinical data: stability, arbitrariness, and fairness](https://mlsanity.com/paper/2604.23954/) (2026)
- [Octopus Protocol: One-Shot Hardware Discovery and Control for AI Agents via Infrastructure-as-Prompts](https://mlsanity.com/paper/2605.09055/) (2026)
- [Verification of Adaptive Agentic Controllers through Finite Rule Revision](https://mlsanity.com/paper/2607.09770/) (2026)

## Abstract (arXiv)

Real-world machine learning systems often encounter model performance degradation due to distributional shifts in the underlying data generating process (DGP). Existing approaches to addressing shifts, such as concept drift adaptation, are limited by their reason-agnostic nature. By choosing from a pre-defined set of actions, such methods implicitly assume that the causes of model degradation are irrelevant to what actions should be taken, limiting their ability to select appropriate adaptations. In this paper, we propose an alternative paradigm to overcome these limitations, called self-healing machine learning (SHML). Contrary to previous approaches, SHML autonomously diagnoses the reason for degradation and proposes diagnosis-based corrective actions. We formalize SHML as an optimization problem over a space of adaptation actions to minimize the expected risk under the shifted DGP. We introduce a theoretical framework for self-healing systems and build an agentic self-healing solution H-LLM which uses large language models to perform self-diagnosis by reasoning about the structure underlying the DGP, and self-adaptation by proposing and evaluating corrective actions. Empirically, we analyze different components of H-LLM to understand why and when it works, demonstrating the potential of self-healing ML.