# How to Sustainably Monitor ML-Enabled Systems? Accuracy and Energy Efficiency Tradeoffs in Concept Drift Detection

arXiv: https://arxiv.org/abs/2404.19452 · HTML: https://mlsanity.com/paper/2404.19452/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 30 Apr 2024.

## Builds on (most cited first, 21 of 21)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift](https://mlsanity.com/paper/1810.11953/) (2018)
- [Challenges in Deploying Machine Learning: a Survey of Case Studies](https://mlsanity.com/paper/2011.09926/) (2020)
- [Predicting with Confidence on Unseen Distributions](https://mlsanity.com/paper/2107.03315/) (2021)
- [Characterizing Concept Drift](https://mlsanity.com/paper/1511.03816/) (2015)
- [A Systematic Review of Green AI](https://mlsanity.com/paper/2301.11047/) (2023)
- [Software Engineering for AI-Based Systems: A Survey](https://mlsanity.com/paper/2105.01984/) (2021)
- [On the Reliable Detection of Concept Drift from Streaming Unlabeled Data](https://mlsanity.com/paper/1704.00023/) (2017)
- [Data-Centric Green AI: An Exploratory Empirical Study](https://mlsanity.com/paper/2204.02766/) (2022)
- [Operationalizing Machine Learning: An Interview Study](https://mlsanity.com/paper/2209.09125/) (2022)
- [A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitioners](https://mlsanity.com/paper/2304.00078/) (2023)
- [A Synthesis of Green Architectural Tactics for ML-Enabled Systems](https://mlsanity.com/paper/2312.09610/) (2023)
- [Energy Efficiency of Training Neural Network Architectures: An Empirical Study](https://mlsanity.com/paper/2302.00967/) (2023)
- [Towards Observability for Production Machine Learning Pipelines](https://mlsanity.com/paper/2108.13557/) (2021)
- [Are Concept Drift Detectors Reliable Alarming Systems? -- A Comparative Study](https://mlsanity.com/paper/2211.13098/) (2022)
- [Detecting Concept Drift in the Presence of Sparsity -- A Case Study of Automated Change Risk Assessment System](https://mlsanity.com/paper/2207.13287/) (2022)

## Built on by (most cited first, 3 of 3)

- [The More the Merrier? Navigating Accuracy vs. Energy Efficiency Design Trade-Offs in Ensemble Learning Systems](https://mlsanity.com/paper/2407.02914/) (2024)
- [Sustainable Machine Learning Retraining: Optimizing Energy Efficiency Without Compromising Accuracy](https://mlsanity.com/paper/2506.13838/) (2025)
- [FMMO: Detecting the Divergence Between Local Attribution and Global Drift](https://mlsanity.com/paper/2609.06173/) (2026)

## Abstract (arXiv)

ML-enabled systems that are deployed in a production environment typically suffer from decaying model prediction quality through concept drift, i.e., a gradual change in the statistical characteristics of a certain real-world domain. To combat this, a simple solution is to periodically retrain ML models, which unfortunately can consume a lot of energy. One recommended tactic to improve energy efficiency is therefore to systematically monitor the level of concept drift and only retrain when it becomes unavoidable. Different methods are available to do this, but we know very little about their concrete impact on the tradeoff between accuracy and energy efficiency, as these methods also consume energy themselves. To address this, we therefore conducted a controlled experiment to study the accuracy vs. energy efficiency tradeoff of seven common methods for concept drift detection. We used five synthetic datasets, each in a version with abrupt and one with gradual drift, and trained six different ML models as base classifiers. Based on a full factorial design, we tested 420 combinations (7 drift detectors * 5 datasets * 2 types of drift * 6 base classifiers) and compared energy consumption and drift detection accuracy. Our results indicate that there are three types of detectors: a) detectors that sacrifice energy efficiency for detection accuracy (KSWIN), b) balanced detectors that consume low to medium energy with good accuracy (HDDM_W, ADWIN), and c) detectors that consume very little energy but are unusable in practice due to very poor accuracy (HDDM_A, PageHinkley, DDM, EDDM). By providing rich evidence for this energy efficiency tactic, our findings support ML practitioners in choosing the best suited method of concept drift detection for their ML-enabled systems.