mlsanityData to 7 Oct 2026

Pi-transformer: A prior-informed dual-attention model for multivariate time-series anomaly detection

arXiv 2509.19985 ↗First posted 24 Sep 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Anomalies in multivariate time series often arise from temporal context and cross-channel coordination rather than isolated outliers. We present Pi-Transformer (Prior-Informed Transformer), a transformer with two attention pathways: data-driven series attention and a smoothly evolving prior attention that encodes temporal invariants such as scale-related self-similarity and phase synchrony. The prior provides an amplitude-insensitive temporal reference that calibrates reconstruction error. During training, we pair a reconstruction objective with a divergence term that encourages agreement between the two attentions while keeping them meaningfully distinct. The prior is regularised to evolve smoothly and is lightly distilled towards dataset-level statistics. At inference, the model combines an alignment-weighted reconstruction signal (Energy) with a mismatch signal that highlights timing and phase disruptions, and fuses them into a single score for detection. Across five benchmarks (SMD, MSL, SMAP, SWaT, and PSM), Pi-Transformer achieves state-of-the-art or highly competitive F1, with particular strength on timing and phase-breaking anomalies. Case analyses show complementary behaviour of the two streams and interpretable detections around regime changes. Embedding prior attention into transformer scoring yields a calibrated and robust approach to anomaly detection in complex multivariate systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 60 references that are in the map. First posted on arXiv 24 Sep 2025.

Builds on

20 of 60 references.

References · 20

  1. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
  2. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
  3. 2018Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholdingcited by 407
  4. 2021Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancycited by 387
  5. 2021Graph Neural Network-Based Anomaly Detection in Multivariate Time Seriescited by 300
  6. 2023SpectralGPT: Spectral Remote Sensing Foundation Modelcited by 250
  7. 2022TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Datacited by 211
  8. 2020A review on outlier/anomaly detection in time series datacited by 187
  9. 2019MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networkscited by 176
  10. 2020Multivariate Time-series Anomaly Detection via Graph Attention Networkcited by 165
  11. 2023DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detectioncited by 103
  12. 2024CATCH: Channel-Aware multivariate Time Series Anomaly Detection via Frequency Patchingcited by 91
8 more
  1. 2023MEMTO: Memory-guided Transformer for Multivariate Time Series Anomaly Detectioncited by 26
  2. 2023AnomalyBERT: Self-Supervised Transformer for Time Series Anomaly Detection using Data Degradation Schemecited by 25
  3. 2022MAD: Self-Supervised Masked Anomaly Detection Task for Multivariate Time Seriescited by 12
  4. 2023LARA: A Light and Anti-overfitting Retraining Approach for Unsupervised Time Series Anomaly Detectioncited by 9
  5. 2023Multivariate Time Series Anomaly Detection via Dynamic Graph Forecastingcited by 7
  6. 2024SimAD: A Simple Dissimilarity-based Approach for Time Series Anomaly Detectioncited by 6
  7. 2025Decomposition-based multi-scale transformer framework for time series anomaly detectioncited by 5
  8. 2024RESTAD: REconstruction and Similarity based Transformer for time series Anomaly Detectioncited by 1

Built on by

Not cited yet.