mlsanityData to 7 Oct 2026

Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy

arXiv 2110.02642 ↗First posted 6 Oct 2021cs.LGcited by 387

Not on the map. Why?

Abstract (arXiv)

Unsupervised detection of anomaly points in time series is a challenging problem, which requires the model to derive a distinguishable criterion. Previous methods tackle the problem mainly through learning pointwise representation or pairwise association, however, neither is sufficient to reason about the intricate dynamics. Recently, Transformers have shown great power in unified modeling of pointwise representation and pairwise association, and we find that the self-attention weight distribution of each time point can embody rich association with the whole series. Our key observation is that due to the rarity of anomalies, it is extremely difficult to build nontrivial associations from abnormal points to the whole series, thereby, the anomalies' associations shall mainly concentrate on their adjacent time points. This adjacent-concentration bias implies an association-based criterion inherently distinguishable between normal and abnormal points, which we highlight through the Association Discrepancy. Technically, we propose the Anomaly Transformer with a new Anomaly-Attention mechanism to compute the association discrepancy. A minimax strategy is devised to amplify the normal-abnormal distinguishability of the association discrepancy. The Anomaly Transformer achieves state-of-the-art results on six unsupervised time series anomaly detection benchmarks of three applications: service monitoring, space & earth exploration, and water treatment.

Key facts. Cited by 387 papers in the map (data through 7 Oct 2026). Builds on 20 of its 49 references that are in the map. First posted on arXiv 6 Oct 2021.

Builds on

20 of 49 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  8. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  9. 2020Reformer: The Efficient Transformercited by 1,658
  10. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  11. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 606
  12. 2018Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholdingcited by 407
8 more
  1. 2021Graph Neural Network-Based Anomaly Detection in Multivariate Time Seriescited by 300
  2. 2018Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applicationscited by 223
  3. 2017A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencodercited by 209
  4. 2019MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networkscited by 176
  5. 2020Multivariate Time-series Anomaly Detection via Graph Attention Networkcited by 165
  6. 2021Learning Graph Structures with Transformer for Multivariate Time Series Anomaly Detection in IoTcited by 104
  7. 2019U-Time: A Fully Convolutional Network for Time Series Segmentation Applied to Sleep Stagingcited by 87
  8. 2020Time Series Change Point Detection with Self-Supervised Contrastive Predictive Codingcited by 58

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 16; 2023: 58; 2024: 87; 2025: 112; 2026: 114.

387 papers.

Sort:

By area:

  1. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
  2. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
  3. 2022Transformers in Time Series: A Surveycited by 489
  4. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
  5. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
  6. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 198
  7. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 191
  8. 2023FITS: Modeling Time Series with 10k Parameterscited by 185
  9. 2023A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detectioncited by 145
  10. 2022Deep Learning for Time Series Anomaly Detection: A Surveycited by 130
  11. 2023LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasterscited by 127
  12. 2024TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysiscited by 113
38 more
  1. 2023SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modelingcited by 112
  2. 2023DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detectioncited by 103
  3. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
  4. 2024UniTS: A Unified Multi-Task Time Series Modelcited by 93
  5. 2024CATCH: Channel-Aware multivariate Time Series Anomaly Detection via Frequency Patchingcited by 91
  6. 2023Time Series as Images: Vision Transformer for Irregularly Sampled Time Seriescited by 79
  7. 2024TSLANet: Rethinking Transformers for Time Series Representation Learningcited by 71
  8. 2024Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysiscited by 65
  9. 2025TAB: Unified Benchmarking of Time Series Anomaly Detection Methodscited by 58
  10. 2024Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classificationcited by 53
  11. 2023A Survey on Time-Series Pre-Trained Modelscited by 52
  12. 2022TFAD: A Decomposition Time Series Anomaly Detection Architecture with Time-Frequency Analysiscited by 48
  13. 2022Transformers in Time-series Analysis: A Tutorialcited by 45
  14. 2022Calibrated One-class Classification for Unsupervised Time Series Anomaly Detectioncited by 39
  15. 2024TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysiscited by 37
  16. 2024Towards a General Time Series Anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoderscited by 37
  17. 2022Self-Supervised Masked Convolutional Transformer Block for Anomaly Detectioncited by 34
  18. 2024TimeDiT: General-purpose Diffusion Transformers for Time Series Foundation Modelcited by 33
  19. 2023Focus the Discrepancy: Intra- and Inter-Correlation Learning for Image Anomaly Detectioncited by 32
  20. 2024Graph Transformers: A Surveycited by 32
  21. 2024Large language models can be zero-shot anomaly detectors for time series?cited by 30
  22. 2024TS-Reasoner: Domain-Oriented Time Series Inference Agents for Reasoning and Automated Analysiscited by 28
  23. 2023MEMTO: Memory-guided Transformer for Multivariate Time Series Anomaly Detectioncited by 26
  24. 2024Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detectioncited by 26
  25. 2024Large Language Models can Deliver Accurate and Interpretable Time Series Anomaly Detectioncited by 26
  26. 2023AnomalyBERT: Self-Supervised Transformer for Time Series Anomaly Detection using Data Degradation Schemecited by 25
  27. 2024Multi-Patch Prediction: Adapting LLMs for Time Series Representation Learningcited by 25
  28. 2025Foundation Models for Spatio-Temporal Data Science: A Tutorial and Surveycited by 23
  29. 2023A Comprehensive Survey of Deep Transfer Learning for Anomaly Detection in Industrial Time Series: Methods, Applications, and Directionscited by 22
  30. 2024RWKV-TS: Beyond Traditional Recurrent Neural Network for Time Series Taskscited by 22
  31. 2024Revisiting VAE for Unsupervised Time Series Anomaly Detection: A Frequency Perspectivecited by 22
  32. 2024TimeSeriesBench: An Industrial-Grade Benchmark for Time Series Anomaly Detection Modelscited by 22
  33. 2022Detecting Multivariate Time Series Anomalies with Zero Known Labelcited by 21
  34. 2024KAN-AD: Time Series Anomaly Detection with Kolmogorov-Arnold Networkscited by 20
  35. 2025Foundation Models for Time Series: A Surveycited by 19
  36. 2024MultiRC: Joint Learning for Time Series Anomaly Prediction and Detection with Multi-scale Reconstructive Contrastcited by 18
  37. 2023Nominality Score Conditioned Time Series Anomaly Detection by Point/Sequential Reconstructioncited by 17
  38. 2023When Model Meets New Normals: Test-time Adaptation for Unsupervised Time-series Anomaly Detectioncited by 17

50 most cited of 387 All 387 (JSON)