# Automatic Model Monitoring for Data Streams

arXiv: https://arxiv.org/abs/1908.04240 · HTML: https://mlsanity.com/paper/1908.04240/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Aug 2019.

## Built on by (most cited first, 9 of 9)

- [Robust Explanation Constraints for Neural Networks](https://mlsanity.com/paper/2212.08507/) (2022)
- [STUDD: A Student-Teacher Method for Unsupervised Concept Drift Detection](https://mlsanity.com/paper/2103.00903/) (2021)
- [MLDemon: Deployment Monitoring for Machine Learning Systems](https://mlsanity.com/paper/2104.13621/) (2021)
- [FairCanary: Rapid Continuous Explainable Fairness](https://mlsanity.com/paper/2106.07057/) (2021)
- [Unified Shapley Framework to Explain Prediction Drift](https://mlsanity.com/paper/2102.07862/) (2021)
- [PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Markets](https://mlsanity.com/paper/2302.00586/) (2023)
- [Data+Shift: Supporting visual investigation of data distribution shifts by data scientists](https://mlsanity.com/paper/2204.14025/) (2022)
- [A Framework for Evaluating and Benchmarking Concept Drift Detection Methods](https://mlsanity.com/paper/2606.07789/) (2026)
- [MUSE: Multi-Tenant Model Serving With Seamless Model Updates](https://mlsanity.com/paper/2602.11776/) (2026)

## Abstract (arXiv)

Detecting concept drift is a well known problem that affects production systems. However, two important issues that are frequently not addressed in the literature are 1) the detection of drift when the labels are not immediately available; and 2) the automatic generation of explanations to identify possible causes for the drift. For example, a fraud detection model in online payments could show a drift due to a hot sale item (with an increase in false positives) or due to a true fraud attack (with an increase in false negatives) before labels are available. In this paper we propose SAMM, an automatic model monitoring system for data streams. SAMM detects concept drift using a time and space efficient unsupervised streaming algorithm and it generates alarm reports with a summary of the events and features that are important to explain it. SAMM was evaluated in five real world fraud detection datasets, each spanning periods up to eight months and totaling more than 22 million online transactions. We evaluated SAMM using human feedback from domain experts, by sending them 100 reports generated by the system. Our results show that SAMM is able to detect anomalous events in a model life cycle that are considered useful by the domain experts. Given these results, SAMM will be rolled out in a next version of Feedzai's Fraud Detection solution.