mlsanityData to 7 Oct 2026

Towards a Rigorous Evaluation of Time-series Anomaly Detection

arXiv 2109.05257 ↗First posted 11 Sep 2021cs.LG cs.AI stat.MEcited by 95

Not on the map. Why?

Abstract (arXiv)

In recent years, proposed studies on time-series anomaly detection (TAD) report high F1 scores on benchmark TAD datasets, giving the impression of clear improvements in TAD. However, most studies apply a peculiar evaluation protocol called point adjustment (PA) before scoring. In this paper, we theoretically and experimentally reveal that the PA protocol has a great possibility of overestimating the detection performance; that is, even a random anomaly score can easily turn into a state-of-the-art TAD method. Therefore, the comparison of TAD methods after applying the PA protocol can lead to misguided rankings. Furthermore, we question the potential of existing TAD methods by showing that an untrained model obtains comparable detection performance to the existing methods even when PA is forbidden. Based on our findings, we propose a new baseline and an evaluation protocol. We expect that our study will help a rigorous evaluation of TAD and lead to further improvement in future researches.

Key facts. Cited by 95 papers in the map (data through 7 Oct 2026). Builds on 13 of its 36 references that are in the map. First posted on arXiv 11 Sep 2021.

Builds on

13 of 36 references.

References · 13

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  3. 2018Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholdingcited by 407
  4. 2019Deep Learning for Anomaly Detection: A Surveycited by 357
  5. 2021Graph Neural Network-Based Anomaly Detection in Multivariate Time Seriescited by 300
  6. 2018Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applicationscited by 223
  7. 2016LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detectioncited by 213
  8. 2020Patch SVDD: Patch-level SVDD for Anomaly Detection and Segmentationcited by 212
  9. 2017A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencodercited by 209
  10. 2018A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Datacited by 181
  11. 2019MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networkscited by 176
  12. 2020Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progresscited by 148
1 more
  1. 2021Learning Graph Structures with Transformer for Multivariate Time Series Anomaly Detection in IoTcited by 104

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 24; 2024: 20; 2025: 20; 2026: 21.

95 papers.

Sort:

By area:

  1. 2022Deep Learning for Time Series Anomaly Detection: A Surveycited by 130
  2. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
  3. 2022Local Evaluation of Time Series Anomaly Detection Algorithmscited by 79
  4. 2025TAB: Unified Benchmarking of Time Series Anomaly Detection Methodscited by 58
  5. 2023A Survey on Time-Series Pre-Trained Modelscited by 52
  6. 2022TFAD: A Decomposition Time Series Anomaly Detection Architecture with Time-Frequency Analysiscited by 48
  7. 2024Universal Time-Series Representation Learning: A Surveycited by 28
  8. 2024Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detectioncited by 26
  9. 2023AnomalyBERT: Self-Supervised Transformer for Time Series Anomaly Detection using Data Degradation Schemecited by 25
  10. 2023Navigating the Metric Maze: A Taxonomy of Evaluation Metrics for Anomaly Detection in Time Seriescited by 24
  11. 2024TimeSeriesBench: An Industrial-Grade Benchmark for Time Series Anomaly Detection Modelscited by 22
  12. 2023T-Rep: Representation Learning for Time Series using Time-Embeddingscited by 19
38 more
  1. 2023Graph Anomaly Detection in Time Series: A Surveycited by 18
  2. 2024MultiRC: Joint Learning for Time Series Anomaly Prediction and Detection with Multi-scale Reconstructive Contrastcited by 18
  3. 2023Correlation-aware Spatial-Temporal Graph Learning for Multivariate Time-series Anomaly Detectioncited by 17
  4. 2023Nominality Score Conditioned Time Series Anomaly Detection by Point/Sequential Reconstructioncited by 17
  5. 2023CARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detectioncited by 15
  6. 2023Explainable Time Series Anomaly Detection using Masked Latent Generative Modelingcited by 15
  7. 2024PATE: Proximity-Aware Time series anomaly Evaluationcited by 13
  8. 2023Unraveling the "Anomaly" in Time Series Anomaly Detection: A Self-supervised Tri-domain Solutioncited by 12
  9. 2023Time Series Anomaly Detection using Diffusion-based Modelscited by 11
  10. 2022Deep Contrastive One-Class Time Series Anomaly Detectioncited by 10
  11. 2023OneShotSTL: One-Shot Seasonal-Trend Decomposition For Online Time Series Anomaly Detection And Forecastingcited by 10
  12. 2023DyEdgeGAT: Dynamic Edge via Graph Attention for Early Fault Detection in IIoT Systemscited by 10
  13. 2024What Do Machine Learning Researchers Mean by "Reproducible"?cited by 8
  14. 2024DACAD: Domain Adaptation Contrastive Learning for Anomaly Detection in Multivariate Time Seriescited by 7
  15. 2025Robust and Explainable Detector of Time Series Anomaly via Augmenting Multiclass Pseudo-Anomaliescited by 7
  16. 2023Multivariate Time Series Anomaly Detection: Fancy Algorithms and Flawed Evaluation Methodologycited by 6
  17. 2023Entropy Causal Graphs for Multivariate Time Series Anomaly Detectioncited by 6
  18. 2024SimAD: A Simple Dissimilarity-based Approach for Time Series Anomaly Detectioncited by 6
  19. 2024TimeInf: Time Series Data Contribution via Influence Functionscited by 6
  20. 2024Vision-Language Models Assisted Unsupervised Video Anomaly Detectioncited by 6
  21. 2023Deep Anomaly Detection on Tennessee Eastman Process Datacited by 5
  22. 2026PaAno: Patch-Based Representation Learning for Time-Series Anomaly Detectioncited by 5
  23. 2022Online Time Series Anomaly Detection with State Space Gaussian Processescited by 4
  24. 2025RoCA: Robust Contrastive One-class Time Series Anomaly Detection with Contaminated Datacited by 4
  25. 2025Structured Temporal Causality for Interpretable Multivariate Time Series Anomaly Detectioncited by 4
  26. 2023AD-NEV: A Scalable Multi-level Neuroevolution Framework for Multivariate Anomaly Detectioncited by 3
  27. 2023Contaminated Multivariate Time-Series Anomaly Detection with Spatio-Temporal Graph Conditional Diffusion Modelscited by 3
  28. 2023Sketching Multidimensional Time Series for Fast Discord Miningcited by 3
  29. 2025Noise-Resilient Point-wise Anomaly Detection in Time Series Using Weak Segment Labelscited by 3
  30. 2022Intrinsic Anomaly Detection for Multi-Variate Time Seriescited by 2
  31. 2022Robust Learning of Deep Time Series Anomaly Detection Models with Contaminated Training Datacited by 2
  32. 2024Edge Conditional Node Update Graph Neural Network for Multi-variate Time Series Anomaly Detectioncited by 2
  33. 2024Joint Selective State Space Model and Detrending for Robust Time Series Anomaly Detectioncited by 2
  34. 2024Self-Supervised Time-Series Anomaly Detection Using Learnable Data Augmentationcited by 2
  35. 2024Unveiling the Flaws: A Critical Analysis of Initialization Effect on Time Series Anomaly Detectioncited by 2
  36. 2024Towards Unbiased Evaluation of Time-series Anomaly Detectorcited by 2
  37. 2024Interpreting Temporal Graph Neural Networks with Koopman Theorycited by 2
  38. 2025PatchTrAD: A Patch-Based Transformer focusing on Patch-Wise Reconstruction Error for Time Series Anomaly Detectioncited by 2

50 most cited of 95 All 95 (JSON)