# Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progress

arXiv: https://arxiv.org/abs/2009.13807 · HTML: https://mlsanity.com/paper/2009.13807/

## Key facts

- Cited by 151 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Sep 2020.

## Built on by (most cited first, 30 of 151)

- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data](https://mlsanity.com/paper/2201.07284/) (2022)
- [Timer: Generative Pre-trained Transformers Are Large Time Series Models](https://mlsanity.com/paper/2402.02368/) (2024)
- [A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection](https://mlsanity.com/paper/2307.03759/) (2023)
- [Towards a Rigorous Evaluation of Time-series Anomaly Detection](https://mlsanity.com/paper/2109.05257/) (2021)
- [Deep Isolation Forest for Anomaly Detection](https://mlsanity.com/paper/2206.06602/) (2022)
- [Local Evaluation of Time Series Anomaly Detection Algorithms](https://mlsanity.com/paper/2206.13167/) (2022)
- [An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series](https://mlsanity.com/paper/2109.11428/) (2021)
- [A Survey on Time-Series Pre-Trained Models](https://mlsanity.com/paper/2305.10716/) (2023)
- [Neural Contextual Anomaly Detection for Time Series](https://mlsanity.com/paper/2107.07702/) (2021)
- [Can LLMs Understand Time Series Anomalies?](https://mlsanity.com/paper/2410.05440/) (2024)
- [Calibrated One-class Classification for Unsupervised Time Series Anomaly Detection](https://mlsanity.com/paper/2207.12201/) (2022)
- [Towards a General Time Series Anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoders](https://mlsanity.com/paper/2405.15273/) (2024)
- [Universal Time-Series Representation Learning: A Survey](https://mlsanity.com/paper/2401.03717/) (2024)
- [Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detection](https://mlsanity.com/paper/2401.15123/) (2024)
- [Unsupervised Model Selection for Time-series Anomaly Detection](https://mlsanity.com/paper/2210.01078/) (2022)
- [Mantis: Lightweight Foundation Model for Time Series Classification](https://mlsanity.com/paper/2502.15637/) (2025)
- [TimeSeriesBench: An Industrial-Grade Benchmark for Time Series Anomaly Detection Models](https://mlsanity.com/paper/2402.10802/) (2024)
- [KAN-AD: Time Series Anomaly Detection with Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2411.00278/) (2024)
- [Detecting Multivariate Time Series Anomalies with Zero Known Label](https://mlsanity.com/paper/2208.02108/) (2022)
- [Do Deep Neural Networks Contribute to Multivariate Time Series Anomaly Detection?](https://mlsanity.com/paper/2204.01637/) (2022)
- [Graph Anomaly Detection in Time Series: A Survey](https://mlsanity.com/paper/2302.00058/) (2023)
- [Correlation-aware Spatial-Temporal Graph Learning for Multivariate Time-series Anomaly Detection](https://mlsanity.com/paper/2307.08390/) (2023)
- [Deep Generative model with Hierarchical Latent Factors for Time Series Anomaly Detection](https://mlsanity.com/paper/2202.07586/) (2022)
- [AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges](https://mlsanity.com/paper/2304.04661/) (2023)
- [Explainable Time Series Anomaly Detection using Masked Latent Generative Modeling](https://mlsanity.com/paper/2311.12550/) (2023)
- [Label-Free Multivariate Time Series Anomaly Detection](https://mlsanity.com/paper/2312.11549/) (2023)
- [Online false discovery rate control for anomaly detection in time series](https://mlsanity.com/paper/2112.03196/) (2021)
- [Time-Series Pattern Recognition in Smart Manufacturing Systems: A Literature Review and Ontology](https://mlsanity.com/paper/2301.12495/) (2023)
- [Detecting Anomalies within Time Series using Local Neural Transformations](https://mlsanity.com/paper/2202.03944/) (2022)

## Abstract (arXiv)

Time series anomaly detection has been a perennially important topic in data science, with papers dating back to the 1950s. However, in recent years there has been an explosion of interest in this topic, much of it driven by the success of deep learning in other domains and for other time series tasks. Most of these papers test on one or more of a handful of popular benchmark datasets, created by Yahoo, Numenta, NASA, etc. In this work we make a surprising claim. The majority of the individual exemplars in these datasets suffer from one or more of four flaws. Because of these four flaws, we believe that many published comparisons of anomaly detection algorithms may be unreliable, and more importantly, much of the apparent progress in recent years may be illusionary. In addition to demonstrating these claims, with this paper we introduce the UCR Time Series Anomaly Archive. We believe that this resource will perform a similar role as the UCR Time Series Classification Archive, by providing the community with a benchmark that allows meaningful comparisons between approaches and a meaningful gauge of overall progress.