# Towards Computational Performance Engineering for Unsupervised Concept Drift Detection -- Complexities, Benchmarking, Performance Analysis

arXiv: https://arxiv.org/abs/2304.08319 · HTML: https://mlsanity.com/paper/2304.08319/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 64 references that are in the map.
- First posted on arXiv 17 Apr 2023.

## Builds on (most cited first, 9 of 9)

- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [From Amateurs to Connoisseurs: Modeling the Evolution of User Expertise through Online Reviews](https://mlsanity.com/paper/1303.4402/) (2013)
- [Characterizing Concept Drift](https://mlsanity.com/paper/1511.03816/) (2015)
- [Challenges in Benchmarking Stream Learning Algorithms with Real-world Data](https://mlsanity.com/paper/2005.00113/) (2020)
- [On the Reliable Detection of Concept Drift from Streaming Unlabeled Data](https://mlsanity.com/paper/1704.00023/) (2017)
- [STUDD: A Student-Teacher Method for Unsupervised Concept Drift Detection](https://mlsanity.com/paper/2103.00903/) (2021)
- [Normalizing Flow based Feature Synthesis for Outlier-Aware Object Detection](https://mlsanity.com/paper/2302.07106/) (2023)
- [Advanced Python Performance Monitoring with Score-P](https://mlsanity.com/paper/2010.15444/) (2020)
- [Labelling Drifts in a Fault Detection System for Wind Turbine Maintenance](https://mlsanity.com/paper/2106.09951/) (2021)

## Built on by (most cited first, 2 of 2)

- [Unsupervised Concept Drift Detection from Deep Learning Representations in Real-time](https://mlsanity.com/paper/2406.17813/) (2024)
- [Modyn: Data-Centric Machine Learning Pipeline Orchestration](https://mlsanity.com/paper/2312.06254/) (2023)

## Abstract (arXiv)

Concept drift detection is crucial for many AI systems to ensure the system's reliability. These systems often have to deal with large amounts of data or react in real-time. Thus, drift detectors must meet computational requirements or constraints with a comprehensive performance evaluation. However, so far, the focus of developing drift detectors is on inference quality, e.g. accuracy, but not on computational performance, such as runtime. Many of the previous works consider computational performance only as a secondary objective and do not have a benchmark for such evaluation. Hence, we propose and explain performance engineering for unsupervised concept drift detection that reflects on computational complexities, benchmarking, and performance analysis. We provide the computational complexities of existing unsupervised drift detectors and discuss why further computational performance investigations are required. Hence, we state and substantiate the aspects of a benchmark for unsupervised drift detection reflecting on inference quality and computational performance. Furthermore, we demonstrate performance analysis practices that have proven their effectiveness in High-Performance Computing, by tracing two drift detectors and displaying their performance data.