# Navigating the Metric Maze: A Taxonomy of Evaluation Metrics for Anomaly Detection in Time Series

arXiv: https://arxiv.org/abs/2303.01272 · HTML: https://mlsanity.com/paper/2303.01272/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 97 references that are in the map.
- First posted on arXiv 2 Mar 2023.

## Builds on (most cited first, 30 of 34)

- [Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding](https://mlsanity.com/paper/1802.04431/) (2018)
- [Graph Neural Network-Based Anomaly Detection in Multivariate Time Series](https://mlsanity.com/paper/2106.06947/) (2021)
- [Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications](https://mlsanity.com/paper/1802.03903/) (2018)
- [TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data](https://mlsanity.com/paper/2201.07284/) (2022)
- [A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencoder](https://mlsanity.com/paper/1711.00614/) (2017)
- [A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data](https://mlsanity.com/paper/1811.08055/) (2018)
- [MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks](https://mlsanity.com/paper/1901.04997/) (2019)
- [Multivariate Time-series Anomaly Detection via Graph Attention Network](https://mlsanity.com/paper/2009.02040/) (2020)
- [Time-Series Anomaly Detection Service at Microsoft](https://mlsanity.com/paper/1906.03821/) (2019)
- [Deep Anomaly Detection with Deviation Networks](https://mlsanity.com/paper/1911.08623/) (2019)
- [Learning Graph Structures with Transformer for Multivariate Time Series Anomaly Detection in IoT](https://mlsanity.com/paper/2104.03466/) (2021)
- [Towards a Rigorous Evaluation of Time-series Anomaly Detection](https://mlsanity.com/paper/2109.05257/) (2021)
- [Evaluating Real-time Anomaly Detection Algorithms - the Numenta Anomaly Benchmark](https://mlsanity.com/paper/1510.03336/) (2015)
- [Local Evaluation of Time Series Anomaly Detection Algorithms](https://mlsanity.com/paper/2206.13167/) (2022)
- [An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series](https://mlsanity.com/paper/2109.11428/) (2021)
- [Precision and Recall for Time Series](https://mlsanity.com/paper/1803.03639/) (2018)
- [Anomaly Detection in Univariate Time-series: A Survey on the State-of-the-Art](https://mlsanity.com/paper/2004.00433/) (2020)
- [Exathlon: A Benchmark for Explainable Anomaly Detection over Time Series](https://mlsanity.com/paper/2010.05073/) (2020)
- [TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks](https://mlsanity.com/paper/2009.07769/) (2020)
- [Graph-Augmented Normalizing Flows for Anomaly Detection of Multiple Time Series](https://mlsanity.com/paper/2202.07857/) (2022)
- [Unsupervised and Semi-supervised Anomaly Detection with LSTM Neural Networks](https://mlsanity.com/paper/1710.09207/) (2017)
- [Calibrated One-class Classification for Unsupervised Time Series Anomaly Detection](https://mlsanity.com/paper/2207.12201/) (2022)
- [Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder](https://mlsanity.com/paper/2102.01331/) (2021)
- [Unsupervised Time Series Outlier Detection with Diversity-Driven Convolutional Ensembles -- Extended Version](https://mlsanity.com/paper/2111.11108/) (2021)
- [Time Series Anomaly Detection for Cyber-Physical Systems via Neural System Identification and Bayesian Filtering](https://mlsanity.com/paper/2106.07992/) (2021)
- [TAnoGAN: Time Series Anomaly Detection with Generative Adversarial Networks](https://mlsanity.com/paper/2008.09567/) (2020)
- [Unsupervised Model Selection for Time-series Anomaly Detection](https://mlsanity.com/paper/2210.01078/) (2022)
- [MSTREAM: Fast Anomaly Detection in Multi-Aspect Streams](https://mlsanity.com/paper/2009.08451/) (2020)
- [Deep Generative model with Hierarchical Latent Factors for Time Series Anomaly Detection](https://mlsanity.com/paper/2202.07586/) (2022)
- [TiSAT: Time Series Anomaly Transformer](https://mlsanity.com/paper/2203.05167/) (2022)

## Built on by (most cited first, 24 of 24)

- [Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects](https://mlsanity.com/paper/2306.10125/) (2023)
- [VUS: Effective and Efficient Accuracy Measures for Time-Series Anomaly Detection](https://mlsanity.com/paper/2502.13318/) (2025)
- [Time Series Anomaly Detection using Diffusion-based Models](https://mlsanity.com/paper/2311.01452/) (2023)
- [Anomaly Detection and Generation with Diffusion Models: A Survey](https://mlsanity.com/paper/2506.09368/) (2025)
- [SimAD: A Simple Dissimilarity-based Approach for Time Series Anomaly Detection](https://mlsanity.com/paper/2405.11238/) (2024)
- [We Need to Rethink Benchmarking in Anomaly Detection](https://mlsanity.com/paper/2507.15584/) (2025)
- [SoftED: Metrics for Soft Evaluation of Time Series Event Detection](https://mlsanity.com/paper/2304.00439/) (2023)
- [Towards Unbiased Evaluation of Time-series Anomaly Detector](https://mlsanity.com/paper/2409.13053/) (2024)
- [Federated Learning for Multivariate Time Series Anomaly Detection in Industrial Automation](https://mlsanity.com/paper/2605.27486/) (2026)
- [Exploring the Impact of Outlier Variability on Anomaly Detection Evaluation Metrics](https://mlsanity.com/paper/2409.15986/) (2024)
- [A Problem-Oriented Taxonomy of Evaluation Metrics for Time Series Anomaly Detection](https://mlsanity.com/paper/2511.18739/) (2025)
- [Topological Analysis for Detecting Anomalies (TADA) in Time Series](https://mlsanity.com/paper/2406.06168/) (2024)
- [ALPEC: A Comprehensive Evaluation Framework and Dataset for Machine Learning-Based Arousal Detection in Clinical Practice](https://mlsanity.com/paper/2409.13367/) (2024)
- [Detecting Anomalies Using Rotated Isolation Forest](https://mlsanity.com/paper/2501.17787/) (2025)
- [OIPR: Evaluation for Time-series Anomaly Detection Inspired by Operator Interest](https://mlsanity.com/paper/2503.01260/) (2025)
- [CCE: Confidence-Consistency Evaluation for Time Series Anomaly Detection](https://mlsanity.com/paper/2509.01098/) (2025)
- [Forecasting Anomaly Precursors via Uncertainty-Aware Time-Series Ensembles](https://mlsanity.com/paper/2602.17028/) (2026)
- [Temporal-Conditioned Normalizing Flows for Multivariate Time Series Anomaly Detection](https://mlsanity.com/paper/2603.09490/) (2026)
- [Anomaly detection in time-series via inductive biases in the latent space of conditional normalizing flows](https://mlsanity.com/paper/2603.11756/) (2026)
- [Weighted Score-Oriented Losses for Temporally Localized Event Prediction](https://mlsanity.com/paper/2606.23145/) (2026)
- [Did We Actually Fix It? An Independent Adversarial Stress-Test of Post-Point-Adjustment Evaluation Metrics for Time-Series Anomaly Detection](https://mlsanity.com/paper/2607.11969/) (2026)
- [ClouDens: Operational Context-Aware Anomaly Detection for Large-scale Cloud System Monitoring](https://mlsanity.com/paper/2607.18127/) (2026)
- [An Insight on Evaluation Metrics Under the Imbalanced Case of Anomaly Detection](https://mlsanity.com/paper/2607.22286/) (2026)
- [In a Streaming World, Should You Stand Still? A Comprehensive Benchmark of Anomaly Detection in Streams](https://mlsanity.com/paper/2609.39215/) (2026)

## Abstract (arXiv)

The field of time series anomaly detection is constantly advancing, with several methods available, making it a challenge to determine the most appropriate method for a specific domain. The evaluation of these methods is facilitated by the use of metrics, which vary widely in their properties. Despite the existence of new evaluation metrics, there is limited agreement on which metrics are best suited for specific scenarios and domain, and the most commonly used metrics have faced criticism in the literature. This paper provides a comprehensive overview of the metrics used for the evaluation of time series anomaly detection methods, and also defines a taxonomy of these based on how they are calculated. By defining a set of properties for evaluation metrics and a set of specific case studies and experiments, twenty metrics are analyzed and discussed in detail, highlighting the unique suitability of each for specific tasks. Through extensive experimentation and analysis, this paper argues that the choice of evaluation metric must be made with care, taking into account the specific requirements of the task at hand.