# Open-Source Drift Detection Tools in Action: Insights from Two Use Cases

arXiv: https://arxiv.org/abs/2404.18673 · HTML: https://mlsanity.com/paper/2404.18673/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 8 references that are in the map.
- First posted on arXiv 29 Apr 2024.

## Builds on (most cited first, 4 of 4)

- [Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift](https://mlsanity.com/paper/1810.11953/) (2018)
- [From Concept Drift to Model Degradation: An Overview on Performance-Aware Drift Detectors](https://mlsanity.com/paper/2203.11070/) (2022)
- [A Unifying Causal Framework for Analyzing Dataset Shift-stable Learning Algorithms](https://mlsanity.com/paper/1905.11374/) (2019)
- [REIN: A Comprehensive Benchmark Framework for Data Cleaning Methods in ML Pipelines](https://mlsanity.com/paper/2302.04702/) (2023)

## Built on by (most cited first, 3 of 3)

- [Predictive maintenance solution for industrial systems -- an unsupervised approach based on log periodic power law](https://mlsanity.com/paper/2408.05231/) (2024)
- [A Representation Learning Approach to Feature Drift Detection in Wireless Networks](https://mlsanity.com/paper/2505.10325/) (2025)
- [Beyond Marginal Monitoring: Distributed Joint-Distribution Testing for Data Concept Drift in Large Scale E-Commerce Operations](https://mlsanity.com/paper/2610.08132/) (2026)

## Abstract (arXiv)

Data drifts pose a critical challenge in the lifecycle of machine learning (ML) models, affecting their performance and reliability. In response to this challenge, we present a microbenchmark study, called D3Bench, which evaluates the efficacy of open-source drift detection tools. D3Bench examines the capabilities of Evidently AI, NannyML, and Alibi-Detect, leveraging real-world data from two smart building use cases.We prioritize assessing the functional suitability of these tools to identify and analyze data drifts. Furthermore, we consider a comprehensive set of non-functional criteria, such as the integrability with ML pipelines, the adaptability to diverse data types, user-friendliness, computational efficiency, and resource demands. Our findings reveal that Evidently AI stands out for its general data drift detection, whereas NannyML excels at pinpointing the precise timing of shifts and evaluating their consequent effects on predictive accuracy.