# Statistical Evaluation of Anomaly Detectors for Sequences

arXiv: https://arxiv.org/abs/2008.05788 · HTML: https://mlsanity.com/paper/2008.05788/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2020.

## Built on by (most cited first, 6 of 6)

- [Local Evaluation of Time Series Anomaly Detection Algorithms](https://mlsanity.com/paper/2206.13167/) (2022)
- [Navigating the Metric Maze: A Taxonomy of Evaluation Metrics for Anomaly Detection in Time Series](https://mlsanity.com/paper/2303.01272/) (2023)
- [TimeSeriesBench: An Industrial-Grade Benchmark for Time Series Anomaly Detection Models](https://mlsanity.com/paper/2402.10802/) (2024)
- [Data-driven Design of Context-aware Monitors for Hazard Prediction in Artificial Pancreas Systems](https://mlsanity.com/paper/2104.02545/) (2021)
- [A Problem-Oriented Taxonomy of Evaluation Metrics for Time Series Anomaly Detection](https://mlsanity.com/paper/2511.18739/) (2025)
- [ALPEC: A Comprehensive Evaluation Framework and Dataset for Machine Learning-Based Arousal Detection in Clinical Practice](https://mlsanity.com/paper/2409.13367/) (2024)

## Abstract (arXiv)

Although precision and recall are standard performance measures for anomaly detection, their statistical properties in sequential detection settings are poorly understood. In this work, we formalize a notion of precision and recall with temporal tolerance for point-based anomaly detection in sequential data. These measures are based on time-tolerant confusion matrices that may be used to compute time-tolerant variants of many other standard measures. However, care has to be taken to preserve interpretability. We perform a statistical simulation study to demonstrate that precision and recall may overestimate the performance of a detector, when computed with temporal tolerance. To alleviate this problem, we show how to obtain null distributions for the two measures to assess the statistical significance of reported results.