mlsanityData to 14 Oct 2026

Surprised by Attention: Predictable Query Dynamics for Time Series Anomaly Detection

arXiv 2603.12916 ↗First posted 13 Mar 2026cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Multivariate time series anomalies often manifest as shifts in cross-channel dependencies rather than simple amplitude excursions. In autonomous driving, for instance, a steering command might be internally consistent but decouple from the resulting lateral acceleration. Residual-based detectors can miss such anomalies when flexible sequence models still reconstruct signals plausibly despite altered coordination. We introduce AxonAD, an unsupervised detector that treats multi-head attention query evolution as a short horizon predictable process. A gradient-updated reconstruction pathway is coupled with a history-only predictor that forecasts future query vectors from past context. This is trained via a masked predictor-target objective against an exponential moving average (EMA) target encoder. At inference, reconstruction error is combined with a tail-aggregated query mismatch score, which measures cosine deviation between predicted and target queries on recent timesteps. This dual approach provides sensitivity to structural dependency shifts while retaining amplitude-level detection. On proprietary in-vehicle telemetry with interval annotations and on the TSB-AD multi-variate suite (17 datasets, 180 series) with threshold-free and range-aware metrics, AxonAD improves ranking quality and temporal localization over strong baselines. Ablations confirm that query prediction and combined scoring are the primary drivers of the observed gains. Code is available at the URL https://github.com/iis-esslingen/AxonAD.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 20 of its 37 references that are in the map. First posted on arXiv 13 Mar 2026.

Builds on

20 of 37 references.

References · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Layer Normalizationcited by 5,669
  3. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  4. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  5. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  6. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  7. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  8. 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 508
  9. 2021Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancycited by 394
  10. 2021Graph Neural Network-Based Anomaly Detection in Multivariate Time Seriescited by 301
  11. 2025LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristicscited by 254
  12. 2018Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applicationscited by 224
8 more
  1. 2022TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Datacited by 214
  2. 2023FITS: Modeling Time Series with 10k Parameterscited by 195
  3. 2020COPOD: Copula-Based Outlier Detectioncited by 146
  4. 2018Extended Isolation Forestcited by 81
  5. 2021Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encodercited by 39
  6. 2025VUS: Effective and Efficient Accuracy Measures for Time-Series Anomaly Detectioncited by 13
  7. 2023MA-VAE: Multi-head Attention-based Variational Autoencoder Approach for Anomaly Detection in Multivariate Time-series Applied to Automotive Endurance Powertrain Testingcited by 5
  8. 2025Competition and Attraction Improve Model Fusioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.