# Local Evaluation of Time Series Anomaly Detection Algorithms

arXiv: https://arxiv.org/abs/2206.13167 · HTML: https://mlsanity.com/paper/2206.13167/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 30 references that are in the map.
- First posted on arXiv 27 Jun 2022.

## Builds on (most cited first, 12 of 12)

- [Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding](https://mlsanity.com/paper/1802.04431/) (2018)
- [Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications](https://mlsanity.com/paper/1802.03903/) (2018)
- [Time-Series Anomaly Detection Service at Microsoft](https://mlsanity.com/paper/1906.03821/) (2019)
- [Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progress](https://mlsanity.com/paper/2009.13807/) (2020)
- [Towards a Rigorous Evaluation of Time-series Anomaly Detection](https://mlsanity.com/paper/2109.05257/) (2021)
- [Evaluating Real-time Anomaly Detection Algorithms - the Numenta Anomaly Benchmark](https://mlsanity.com/paper/1510.03336/) (2015)
- [An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series](https://mlsanity.com/paper/2109.11428/) (2021)
- [Precision and Recall for Time Series](https://mlsanity.com/paper/1803.03639/) (2018)
- [Greenhouse: A Zero-Positive Machine Learning System for Time-Series Anomaly Detection](https://mlsanity.com/paper/1801.03168/) (2018)
- [Statistical Evaluation of Anomaly Detectors for Sequences](https://mlsanity.com/paper/2008.05788/) (2020)
- [Anomaly Detection for Industrial Control Systems Using Sequence-to-Sequence Neural Networks](https://mlsanity.com/paper/1911.04831/) (2019)
- [Novel semi-metrics for multivariate change point analysis and anomaly detection](https://mlsanity.com/paper/1911.00995/) (2019)

## Built on by (most cited first, 30 of 79)

- [Deep Learning for Time Series Anomaly Detection: A Survey](https://mlsanity.com/paper/2211.05244/) (2022)
- [DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection](https://mlsanity.com/paper/2306.10347/) (2023)
- [Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects](https://mlsanity.com/paper/2306.10125/) (2023)
- [CATCH: Channel-Aware multivariate Time Series Anomaly Detection via Frequency Patching](https://mlsanity.com/paper/2410.12261/) (2024)
- [TAB: Unified Benchmarking of Time Series Anomaly Detection Methods](https://mlsanity.com/paper/2506.18046/) (2025)
- [A Survey on Time-Series Pre-Trained Models](https://mlsanity.com/paper/2305.10716/) (2023)
- [Can LLMs Understand Time Series Anomalies?](https://mlsanity.com/paper/2410.05440/) (2024)
- [Towards a General Time Series Anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoders](https://mlsanity.com/paper/2405.15273/) (2024)
- [Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detection](https://mlsanity.com/paper/2401.15123/) (2024)
- [Navigating the Metric Maze: A Taxonomy of Evaluation Metrics for Anomaly Detection in Time Series](https://mlsanity.com/paper/2303.01272/) (2023)
- [TimeSeriesBench: An Industrial-Grade Benchmark for Time Series Anomaly Detection Models](https://mlsanity.com/paper/2402.10802/) (2024)
- [KAN-AD: Time Series Anomaly Detection with Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2411.00278/) (2024)
- [MultiRC: Joint Learning for Time Series Anomaly Prediction and Detection with Multi-scale Reconstructive Contrast](https://mlsanity.com/paper/2410.15997/) (2024)
- [PATE: Proximity-Aware Time series anomaly Evaluation](https://mlsanity.com/paper/2405.12096/) (2024)
- [Unraveling the "Anomaly" in Time Series Anomaly Detection: A Self-supervised Tri-domain Solution](https://mlsanity.com/paper/2311.11235/) (2023)
- [PatchAD: A Lightweight Patch-based MLP-Mixer for Time Series Anomaly Detection](https://mlsanity.com/paper/2401.09793/) (2024)
- [Can Multimodal LLMs Perform Time Series Anomaly Detection?](https://mlsanity.com/paper/2502.17812/) (2025)
- [CrossAD: Time Series Anomaly Detection with Cross-scale Associations and Cross-window Modeling](https://mlsanity.com/paper/2510.12489/) (2025)
- [Deep Contrastive One-Class Time Series Anomaly Detection](https://mlsanity.com/paper/2207.01472/) (2022)
- [Towards Foundation Models for Zero-Shot Time Series Anomaly Detection: Leveraging Synthetic Data and Relative Context Discrepancy](https://mlsanity.com/paper/2509.21190/) (2025)
- [Multivariate Time Series Anomaly Detection by Capturing Coarse-Grained Intra- and Inter-Variate Dependencies](https://mlsanity.com/paper/2501.16364/) (2025)
- [DACAD: Domain Adaptation Contrastive Learning for Anomaly Detection in Multivariate Time Series](https://mlsanity.com/paper/2404.11269/) (2024)
- [mTSBench: Benchmarking Multivariate Time Series Anomaly Detection and Model Selection at Scale](https://mlsanity.com/paper/2506.21550/) (2025)
- [STAR: Boosting Time Series Foundation Models for Anomaly Detection through State-aware Adapter](https://mlsanity.com/paper/2510.16014/) (2025)
- [SimAD: A Simple Dissimilarity-based Approach for Time Series Anomaly Detection](https://mlsanity.com/paper/2405.11238/) (2024)
- [Online Model-based Anomaly Detection in Multivariate Time Series: Taxonomy, Survey, Research Challenges and Future Directions](https://mlsanity.com/paper/2408.03747/) (2024)
- [ARFBench: Benchmarking Time Series Question Answering Ability for Software Incident Response](https://mlsanity.com/paper/2604.21199/) (2026)
- [Mamba Adaptive Anomaly Transformer with association discrepancy for time series](https://mlsanity.com/paper/2502.07858/) (2025)
- [GenIAS: Generator for Instantiating Anomalies in time Series](https://mlsanity.com/paper/2502.08262/) (2025)
- [RoCA: Robust Contrastive One-class Time Series Anomaly Detection with Contaminated Data](https://mlsanity.com/paper/2503.18385/) (2025)

## Abstract (arXiv)

In recent years, specific evaluation metrics for time series anomaly detection algorithms have been developed to handle the limitations of the classical precision and recall. However, such metrics are heuristically built as an aggregate of multiple desirable aspects, introduce parameters and wipe out the interpretability of the output. In this article, we first highlight the limitations of the classical precision/recall, as well as the main issues of the recent event-based metrics -- for instance, we show that an adversary algorithm can reach high precision and recall on almost any dataset under weak assumption. To cope with the above problems, we propose a theoretically grounded, robust, parameter-free and interpretable extension to precision/recall metrics, based on the concept of ``affiliation'' between the ground truth and the prediction sets. Our metrics leverage measures of duration between ground truth and predictions, and have thus an intuitive interpretation. By further comparison against random sampling, we obtain a normalized precision/recall, quantifying how much a given set of results is better than a random baseline prediction. By construction, our approach keeps the evaluation local regarding ground truth events, enabling fine-grained visualization and interpretation of algorithmic results. We compare our proposal against various public time series anomaly detection datasets, algorithms and metrics. We further derive theoretical properties of the affiliation metrics that give explicit expectations about their behavior and ensure robustness against adversary strategies.