mlsanityData to 14 Oct 2026

An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series

arXiv 2109.11428 ↗First posted 23 Sep 2021cs.LG cs.AI stat.MLcited by 68

Not on the map. Why?

Abstract (arXiv)

Several techniques for multivariate time series anomaly detection have been proposed recently, but a systematic comparison on a common set of datasets and metrics is lacking. This paper presents a systematic and comprehensive evaluation of unsupervised and semi-supervised deep-learning based methods for anomaly detection and diagnosis on multivariate time series data from cyberphysical systems. Unlike previous works, we vary the model and post-processing of model errors, i.e. the scoring functions independently of each other, through a grid of 10 models and 4 scoring functions, comparing these variants to state of the art methods. In time-series anomaly detection, detecting anomalous events is more important than detecting individual anomalous time-points. Through experiments, we find that the existing evaluation metrics either do not take events into account, or cannot distinguish between a good detector and trivial detectors, such as a random or an all-positive detector. We propose a new metric to overcome these drawbacks, namely, the composite F-score (Fc1), for evaluating time-series anomaly detection. Our study highlights that dynamic scoring functions work much better than static ones for multivariate time series anomaly detection, and the choice of scoring functions often matters more than the choice of the underlying model. We also find that a simple, channel-wise model - the Univariate Fully-Connected Auto-Encoder, with the dynamic Gaussian scoring function emerges as a winning candidate for both anomaly detection and diagnosis, beating state of the art algorithms.

Key facts. Cited by 68 papers in the map (data through 14 Oct 2026). Builds on 17 of its 53 references that are in the map. First posted on arXiv 23 Sep 2021.

Builds on

17 of 53 references.

References · 17

  1. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  2. 2018Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholdingcited by 410
  3. 2016Tutorial on Variational Autoencoderscited by 409
  4. 2019Deep Learning for Anomaly Detection: A Surveycited by 358
  5. 2018Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applicationscited by 224
  6. 2019PyOD: A Python Toolbox for Scalable Outlier Detectioncited by 222
  7. 2016LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detectioncited by 215
  8. 2017A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencodercited by 209
  9. 2020A review on outlier/anomaly detection in time series datacited by 189
  10. 2018A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Datacited by 182
  11. 2019MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networkscited by 176
  12. 2019Time-Series Anomaly Detection Service at Microsoftcited by 165
5 more
  1. 2020Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progresscited by 151
  2. 2018One-Class Adversarial Nets for Fraud Detectioncited by 33
  3. 2017Automatic Anomaly Detection in the Cloud Via Statistical Learningcited by 25
  4. 2017Deep Learning for Unsupervised Insider Threat Detection in Structured Cybersecurity Data Streamscited by 19
  5. 2019A Joint Model for IT Operation Series Prediction and Anomaly Detectioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 8; 2023: 16; 2024: 13; 2025: 14; 2026: 16.

68 papers.

Sort:

By area:

  1. 2023A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detectioncited by 145
  2. 2022Local Evaluation of Time Series Anomaly Detection Algorithmscited by 79
  3. 2022TFAD: A Decomposition Time Series Anomaly Detection Architecture with Time-Frequency Analysiscited by 48
  4. 2022Self-Supervised Time Series Representation Learning via Cross Reconstruction Transformercited by 47
  5. 2022Calibrated One-class Classification for Unsupervised Time Series Anomaly Detectioncited by 40
  6. 2023Navigating the Metric Maze: A Taxonomy of Evaluation Metrics for Anomaly Detection in Time Seriescited by 24
  7. 2024TimeSeriesBench: An Industrial-Grade Benchmark for Time Series Anomaly Detection Modelscited by 23
  8. 2022Do Deep Neural Networks Contribute to Multivariate Time Series Anomaly Detection?cited by 19
  9. 2023Graph Anomaly Detection in Time Series: A Surveycited by 18
  10. 2023Correlation-aware Spatial-Temporal Graph Learning for Multivariate Time-series Anomaly Detectioncited by 18
  11. 2023Nominality Score Conditioned Time Series Anomaly Detection by Point/Sequential Reconstructioncited by 17
  12. 2022Are we certain it's anomalous?cited by 13
38 more
  1. 2024Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Valuescited by 10
  2. 2024Sub-Adjacent Transformer: Improving Time Series Anomaly Detection with Reconstruction Error from Sub-Adjacent Neighborhoodscited by 10
  3. 2025mTSBench: Benchmarking Multivariate Time Series Anomaly Detection and Model Selection at Scalecited by 7
  4. 2022FedTADBench: Federated Time-Series Anomaly Detection Benchmarkcited by 6
  5. 2023Multivariate Time Series Anomaly Detection: Fancy Algorithms and Flawed Evaluation Methodologycited by 6
  6. 2023Entropy Causal Graphs for Multivariate Time Series Anomaly Detectioncited by 6
  7. 2024SimAD: A Simple Dissimilarity-based Approach for Time Series Anomaly Detectioncited by 6
  8. 2026ARFBench: Benchmarking Time Series Question Answering Ability for Software Incident Responsecited by 6
  9. 2022Contrastive predictive coding for Anomaly Detection in Multi-variate Time Series Datacited by 4
  10. 2025Incremental Causal Graph Learning for Online Cyberattack Detection in Cyber-Physical Infrastructurescited by 4
  11. 2023AD-NEV: A Scalable Multi-level Neuroevolution Framework for Multivariate Anomaly Detectioncited by 3
  12. 2023Coupled Attention Networks for Multivariate Time Series Anomaly Detectioncited by 3
  13. 2026Anomalies in Multivariate Time Series Benchmarks Are Mostly Univariatecited by 3
  14. 2021Fast and scalable neuroevolution deep learning architecture search for multivariate anomaly detectioncited by 2
  15. 2023A Survey of Time Series Anomaly Detection Methods in the AIOps Domaincited by 2
  16. 2024Unveiling the Flaws: A Critical Analysis of Initialization Effect on Time Series Anomaly Detectioncited by 2
  17. 2024Towards Unbiased Evaluation of Time-series Anomaly Detectorcited by 2
  18. 2025Possibility for Proactive Anomaly Detectioncited by 2
  19. 2026Federated Learning for Multivariate Time Series Anomaly Detection in Industrial Automationcited by 2
  20. 2023Harnessing Contrastive Learning and Neural Transformation for Time Series Anomaly Detectioncited by 1
  21. 2023SEA++: Multi-Graph-based High-Order Sensor Alignment for Multivariate Time-Series Unsupervised Domain Adaptationcited by 1
  22. 2023EdgeConvFormer: Dynamic Graph CNN and Transformer based Anomaly Detection in Multivariate Time Seriescited by 1
  23. 2024CARE to Compare: A real-world dataset for anomaly detection in wind turbine datacited by 1
  24. 2024Self-Supervised Spatial-Temporal Normality Learning for Time Series Anomaly Detectioncited by 1
  25. 2024Angel or Devil: Discriminating Hard Samples and Anomaly Contaminations for Unsupervised Time Series Anomaly Detectioncited by 1
  26. 2025Adversarial Sample Generation for Anomaly Detection in Industrial Control Systemscited by 1
  27. 2025Cluster-Aware Causal Mixer for Online Anomaly Detection in Multivariate Time Seriescited by 1
  28. 2025TriP-LLM: A Tri-Branch Patch-wise Large Language Model Framework for Time-Series Anomaly Detectioncited by 1
  29. 2025A Problem-Oriented Taxonomy of Evaluation Metrics for Time Series Anomaly Detectioncited by 1
  30. 2026Anomaly Detection for Sparse and Irregular Multivariate Time Series with Latent SDEscited by 1
  31. 2023Towards Meaningful Anomaly Detection: The Effect of Counterfactual Explanations on the Investigation of Anomalies in Multivariate Time Seriescited by 0
  32. 2023Time series anomaly detection with reconstruction-based state-space modelscited by 0
  33. 2023Diagnosis driven Anomaly Detection for CPScited by 0
  34. 2024Unsupervised Distance Metric Learning for Anomaly Detection Over Multivariate Time Seriescited by 0
  35. 2024Towards efficient deep autoencoders for multivariate time series anomaly detectioncited by 0
  36. 2024AD-NEv++ : The multi-architecture neuroevolution-based multivariate anomaly detection frameworkcited by 0
  37. 2024Constraint Guided AutoEncoders for Joint Optimization of Condition Indicator Estimation and Anomaly Detection in Machine Condition Monitoringcited by 0
  38. 2025Federated Koopman-Reservoir Learning for Large-Scale Multivariate Time-Series Anomaly Detectioncited by 0

50 most cited of 68 All 68 (JSON)