# Standardized Evaluation of Machine Learning Methods for Evolving Data Streams

arXiv: https://arxiv.org/abs/2204.13625 · HTML: https://mlsanity.com/paper/2204.13625/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 50 references that are in the map.
- First posted on arXiv 28 Apr 2022.

## Builds on (most cited first, 19 of 19)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [Techniques for Interpretable Machine Learning](https://mlsanity.com/paper/1808.00033/) (2018)
- [River: machine learning for streaming data in Python](https://mlsanity.com/paper/2012.04740/) (2020)
- [Characterizing Concept Drift](https://mlsanity.com/paper/1511.03816/) (2015)
- [CARLA: A Python Library to Benchmark Algorithmic Recourse and Counterfactual Explanation Algorithms](https://mlsanity.com/paper/2108.00783/) (2021)
- [Scikit-Multiflow: A Multi-output Streaming Framework](https://mlsanity.com/paper/1807.04662/) (2018)
- [Challenges in Benchmarking Stream Learning Algorithms with Real-world Data](https://mlsanity.com/paper/2005.00113/) (2020)
- [On the Reliable Detection of Concept Drift from Streaming Unlabeled Data](https://mlsanity.com/paper/1704.00023/) (2017)
- [On Baselines for Local Feature Attributions](https://mlsanity.com/paper/2101.00905/) (2021)
- [Connecting Interpretability and Robustness in Decision Trees through Separation](https://mlsanity.com/paper/2102.07048/) (2021)
- [Learning Parameter Distributions to Detect Concept Drift in Data Streams](https://mlsanity.com/paper/2010.09388/) (2020)
- [Leveraging Model Inherent Variable Importance for Stable Online Feature Selection](https://mlsanity.com/paper/2006.10398/) (2020)
- [Dynamic Model Tree for Interpretable Data Stream Learning](https://mlsanity.com/paper/2203.16181/) (2022)
- [An Eager Splitting Strategy for Online Decision Trees](https://mlsanity.com/paper/2010.10935/) (2020)

## Built on by (most cited first, 4 of 4)

- [Change Detection for Local Explainability in Evolving Data Streams](https://mlsanity.com/paper/2209.02764/) (2022)
- [Offline Handwritten Signature Verification Using a Stream-Based Approach](https://mlsanity.com/paper/2411.06510/) (2024)
- [Evaluating the Efficacy of Instance Incremental vs. Batch Learning in Delayed Label Environments: An Empirical Study on Tabular Data Streaming for Fraud Detection](https://mlsanity.com/paper/2409.10111/) (2024)
- [Know Your Streams: On the Conceptualization, Characterization, and Generation of Intentional Event Streams](https://mlsanity.com/paper/2604.01440/) (2026)

## Abstract (arXiv)

Due to the unspecified and dynamic nature of data streams, online machine learning requires powerful and flexible solutions. However, evaluating online machine learning methods under realistic conditions is difficult. Existing work therefore often draws on different heuristics and simulations that do not necessarily produce meaningful and reliable results. Indeed, in the absence of common evaluation standards, it often remains unclear how online learning methods will perform in practice or in comparison to similar work. In this paper, we propose a comprehensive set of properties for high-quality machine learning in evolving data streams. In particular, we discuss sensible performance measures and evaluation strategies for online predictive modelling, online feature selection and concept drift detection. As one of the first works, we also look at the interpretability of online learning methods. The proposed evaluation standards are provided in a new Python framework called float. Float is completely modular and allows the simultaneous integration of common libraries, such as scikit-multiflow or river, with custom code. Float is open-sourced and can be accessed at https://github.com/haugjo/float. In this sense, we hope that our work will contribute to more standardized, reliable and realistic testing and comparison of online machine learning methods.