# Are Concept Drift Detectors Reliable Alarming Systems? -- A Comparative Study

arXiv: https://arxiv.org/abs/2211.13098 · HTML: https://mlsanity.com/paper/2211.13098/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 45 references that are in the map.
- First posted on arXiv 23 Nov 2022.

## Builds on (most cited first, 4 of 4)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [From Concept Drift to Model Degradation: An Overview on Performance-Aware Drift Detectors](https://mlsanity.com/paper/2203.11070/) (2022)
- [Adaptive XGBoost for Evolving Data Streams](https://mlsanity.com/paper/2005.07353/) (2020)

## Built on by (most cited first, 7 of 7)

- [AI Risk Management Should Incorporate Both Safety and Security](https://mlsanity.com/paper/2405.19524/) (2024)
- [A comprehensive analysis of concept drift locality in data streams](https://mlsanity.com/paper/2311.06396/) (2023)
- [Test & Evaluation Best Practices for Machine Learning-Enabled Systems](https://mlsanity.com/paper/2310.06800/) (2023)
- [How to Sustainably Monitor ML-Enabled Systems? Accuracy and Energy Efficiency Tradeoffs in Concept Drift Detection](https://mlsanity.com/paper/2404.19452/) (2024)
- [McUDI: Model-Centric Unsupervised Degradation Indicator for Failure Prediction AIOps Solutions](https://mlsanity.com/paper/2401.14093/) (2024)
- [Sustainable Machine Learning Retraining: Optimizing Energy Efficiency Without Compromising Accuracy](https://mlsanity.com/paper/2506.13838/) (2025)
- [SCCM : Stream Cruise Control Method for Automated Drift Detection and Adaptation](https://mlsanity.com/paper/2609.09432/) (2026)

## Abstract (arXiv)

As machine learning models increasingly replace traditional business logic in the production system, their lifecycle management is becoming a significant concern. Once deployed into production, the machine learning models are constantly evaluated on new streaming data. Given the continuous data flow, shifting data, also known as concept drift, is ubiquitous in such settings. Concept drift usually impacts the performance of machine learning models, thus, identifying the moment when concept drift occurs is required. Concept drift is identified through concept drift detectors. In this work, we assess the reliability of concept drift detectors to identify drift in time by exploring how late are they reporting drifts and how many false alarms are they signaling. We compare the performance of the most popular drift detectors belonging to two different concept drift detector groups, error rate-based detectors and data distribution-based detectors. We assess their performance on both synthetic and real-world data. In the case of synthetic data, we investigate the performance of detectors to identify two types of concept drift, abrupt and gradual. Our findings aim to help practitioners understand which drift detector should be employed in different situations and, to achieve this, we share a list of the most important observations made throughout this study, which can serve as guidelines for practical usage. Furthermore, based on our empirical results, we analyze the suitability of each concept drift detection group to be used as alarming system.