mlsanityData to 7 Oct 2026

Navigating the Metric Maze: A Taxonomy of Evaluation Metrics for Anomaly Detection in Time Series

arXiv 2303.01272 ↗First posted 2 Mar 2023cs.LGcited by 24

Not on the map. Why?

Abstract (arXiv)

The field of time series anomaly detection is constantly advancing, with several methods available, making it a challenge to determine the most appropriate method for a specific domain. The evaluation of these methods is facilitated by the use of metrics, which vary widely in their properties. Despite the existence of new evaluation metrics, there is limited agreement on which metrics are best suited for specific scenarios and domain, and the most commonly used metrics have faced criticism in the literature. This paper provides a comprehensive overview of the metrics used for the evaluation of time series anomaly detection methods, and also defines a taxonomy of these based on how they are calculated. By defining a set of properties for evaluation metrics and a set of specific case studies and experiments, twenty metrics are analyzed and discussed in detail, highlighting the unique suitability of each for specific tasks. Through extensive experimentation and analysis, this paper argues that the choice of evaluation metric must be made with care, taking into account the specific requirements of the task at hand.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 34 of its 97 references that are in the map. First posted on arXiv 2 Mar 2023.

Builds on

34 of 97 references.

References · 34

  1. 2018Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholdingcited by 407
  2. 2021Graph Neural Network-Based Anomaly Detection in Multivariate Time Seriescited by 300
  3. 2018Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applicationscited by 223
  4. 2022TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Datacited by 211
  5. 2017A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencodercited by 209
  6. 2018A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Datacited by 181
  7. 2019MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networkscited by 176
  8. 2020Multivariate Time-series Anomaly Detection via Graph Attention Networkcited by 165
  9. 2019Time-Series Anomaly Detection Service at Microsoftcited by 163
  10. 2019Deep Anomaly Detection with Deviation Networkscited by 152
  11. 2021Learning Graph Structures with Transformer for Multivariate Time Series Anomaly Detection in IoTcited by 104
  12. 2021Towards a Rigorous Evaluation of Time-series Anomaly Detectioncited by 95
22 more
  1. 2015Evaluating Real-time Anomaly Detection Algorithms - the Numenta Anomaly Benchmarkcited by 87
  2. 2022Local Evaluation of Time Series Anomaly Detection Algorithmscited by 79
  3. 2021An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Seriescited by 68
  4. 2018Precision and Recall for Time Seriescited by 67
  5. 2020Exathlon: A Benchmark for Explainable Anomaly Detection over Time Seriescited by 56
  6. 2020Anomaly Detection in Univariate Time-series: A Survey on the State-of-the-Artcited by 55
  7. 2020TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networkscited by 55
  8. 2022Graph-Augmented Normalizing Flows for Anomaly Detection of Multiple Time Seriescited by 42
  9. 2017Unsupervised and Semi-supervised Anomaly Detection with LSTM Neural Networkscited by 40
  10. 2021Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encodercited by 39
  11. 2022Calibrated One-class Classification for Unsupervised Time Series Anomaly Detectioncited by 39
  12. 2021Unsupervised Time Series Outlier Detection with Diversity-Driven Convolutional Ensembles -- Extended Versioncited by 35
  13. 2021Time Series Anomaly Detection for Cyber-Physical Systems via Neural System Identification and Bayesian Filteringcited by 31
  14. 2020TAnoGAN: Time Series Anomaly Detection with Generative Adversarial Networkscited by 29
  15. 2022Unsupervised Model Selection for Time-series Anomaly Detectioncited by 24
  16. 2020MSTREAM: Fast Anomaly Detection in Multi-Aspect Streamscited by 17
  17. 2022Deep Generative model with Hierarchical Latent Factors for Time Series Anomaly Detectioncited by 16
  18. 2022TiSAT: Time Series Anomaly Transformercited by 15
  19. 2022Are we certain it's anomalous?cited by 13
  20. 2021Time Series Anomaly Detection for Smart Grids: A Surveycited by 12
  21. 2019VELC: A New Variational AutoEncoder Based Model for Time Series Anomaly Detectioncited by 7
  22. 2020Statistical Evaluation of Anomaly Detectors for Sequencescited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 7; 2026: 9.

24 papers.

Sort:

By area:

  1. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
  2. 2023Time Series Anomaly Detection using Diffusion-based Modelscited by 11
  3. 2025VUS: Effective and Efficient Accuracy Measures for Time-Series Anomaly Detectioncited by 11
  4. 2025Anomaly Detection and Generation with Diffusion Models: A Surveycited by 8
  5. 2024SimAD: A Simple Dissimilarity-based Approach for Time Series Anomaly Detectioncited by 6
  6. 2025We Need to Rethink Benchmarking in Anomaly Detectioncited by 4
  7. 2023SoftED: Metrics for Soft Evaluation of Time Series Event Detectioncited by 2
  8. 2024Towards Unbiased Evaluation of Time-series Anomaly Detectorcited by 2
  9. 2026Federated Learning for Multivariate Time Series Anomaly Detection in Industrial Automationcited by 2
  10. 2024Exploring the Impact of Outlier Variability on Anomaly Detection Evaluation Metricscited by 1
  11. 2025A Problem-Oriented Taxonomy of Evaluation Metrics for Time Series Anomaly Detectioncited by 1
  12. 2024Topological Analysis for Detecting Anomalies (TADA) in Time Seriescited by 0
12 more
  1. 2024ALPEC: A Comprehensive Evaluation Framework and Dataset for Machine Learning-Based Arousal Detection in Clinical Practicecited by 0
  2. 2025Detecting Anomalies Using Rotated Isolation Forestcited by 0
  3. 2025OIPR: Evaluation for Time-series Anomaly Detection Inspired by Operator Interestcited by 0
  4. 2025CCE: Confidence-Consistency Evaluation for Time Series Anomaly Detectioncited by 0
  5. 2026Forecasting Anomaly Precursors via Uncertainty-Aware Time-Series Ensemblescited by 0
  6. 2026Temporal-Conditioned Normalizing Flows for Multivariate Time Series Anomaly Detectioncited by 0
  7. 2026Anomaly detection in time-series via inductive biases in the latent space of conditional normalizing flowscited by 0
  8. 2026Weighted Score-Oriented Losses for Temporally Localized Event Predictioncited by 0
  9. 2026Did We Actually Fix It? An Independent Adversarial Stress-Test of Post-Point-Adjustment Evaluation Metrics for Time-Series Anomaly Detectioncited by 0
  10. 2026ClouDens: Operational Context-Aware Anomaly Detection for Large-scale Cloud System Monitoringcited by 0
  11. 2026An Insight on Evaluation Metrics Under the Imbalanced Case of Anomaly Detectioncited by 0
  12. 2026In a Streaming World, Should You Stand Still? A Comprehensive Benchmark of Anomaly Detection in Streamscited by 0