# Sub-Adjacent Transformer: Improving Time Series Anomaly Detection with Reconstruction Error from Sub-Adjacent Neighborhoods

arXiv: https://arxiv.org/abs/2404.18948 · HTML: https://mlsanity.com/paper/2404.18948/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 49 references that are in the map.
- First posted on arXiv 27 Apr 2024.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding](https://mlsanity.com/paper/1802.04431/) (2018)
- [Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy](https://mlsanity.com/paper/2110.02642/) (2021)
- [Graph Neural Network-Based Anomaly Detection in Multivariate Time Series](https://mlsanity.com/paper/2106.06947/) (2021)
- [TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data](https://mlsanity.com/paper/2201.07284/) (2022)
- [A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencoder](https://mlsanity.com/paper/1711.00614/) (2017)
- [A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data](https://mlsanity.com/paper/1811.08055/) (2018)
- [FLatten Transformer: Vision Transformer using Focused Linear Attention](https://mlsanity.com/paper/2308.00442/) (2023)
- [Multivariate Time-series Anomaly Detection via Graph Attention Network](https://mlsanity.com/paper/2009.02040/) (2020)
- [Learning Graph Structures with Transformer for Multivariate Time Series Anomaly Detection in IoT](https://mlsanity.com/paper/2104.03466/) (2021)
- [An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series](https://mlsanity.com/paper/2109.11428/) (2021)
- [Hydra Attention: Efficient Attention with Many Heads](https://mlsanity.com/paper/2209.07484/) (2022)
- [Adjusting for Autocorrelated Errors in Neural Networks for Time Series](https://mlsanity.com/paper/2101.12578/) (2021)
- [Nominality Score Conditioned Time Series Anomaly Detection by Point/Sequential Reconstruction](https://mlsanity.com/paper/2310.15416/) (2023)
- [Time Series Anomaly Detection using Diffusion-based Models](https://mlsanity.com/paper/2311.01452/) (2023)
- [A Comparative Study on Unsupervised Anomaly Detection for Time Series: Experiments and Analysis](https://mlsanity.com/paper/2209.04635/) (2022)

## Built on by (most cited first, 10 of 10)

- [DBLoss: Decomposition-based Loss Function for Time Series Forecasting](https://mlsanity.com/paper/2510.23672/) (2025)
- [Enhancing Time Series Forecasting through Selective Representation Spaces: A Patch Perspective](https://mlsanity.com/paper/2510.14510/) (2025)
- [OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain](https://mlsanity.com/paper/2505.08550/) (2025)
- [FreEformer: Frequency Enhanced Transformer for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2501.13989/) (2025)
- [PaAno: Patch-Based Representation Learning for Time-Series Anomaly Detection](https://mlsanity.com/paper/2602.01359/) (2026)
- [ScatterAD: Temporal-Topological Scattering Mechanism for Time Series Anomaly Detection](https://mlsanity.com/paper/2509.24414/) (2025)
- [Cluster-Aware Causal Mixer for Online Anomaly Detection in Multivariate Time Series](https://mlsanity.com/paper/2506.00188/) (2025)
- [vLinear: A Powerful Linear Model for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2601.13768/) (2026)
- [Selective Denoising Diffusion Model for Time Series Anomaly Detection](https://mlsanity.com/paper/2602.23662/) (2026)
- [Beyond Empirical Support: Structured Outlier Generation via Sinkhorn Optimal Transport](https://mlsanity.com/paper/2609.31470/) (2026)

## Abstract (arXiv)

In this paper, we present the Sub-Adjacent Transformer with a novel attention mechanism for unsupervised time series anomaly detection. Unlike previous approaches that rely on all the points within some neighborhood for time point reconstruction, our method restricts the attention to regions not immediately adjacent to the target points, termed sub-adjacent neighborhoods. Our key observation is that owing to the rarity of anomalies, they typically exhibit more pronounced differences from their sub-adjacent neighborhoods than from their immediate vicinities. By focusing the attention on the sub-adjacent areas, we make the reconstruction of anomalies more challenging, thereby enhancing their detectability. Technically, our approach concentrates attention on the non-diagonal areas of the attention matrix by enlarging the corresponding elements in the training stage. To facilitate the implementation of the desired attention matrix pattern, we adopt linear attention because of its flexibility and adaptability. Moreover, a learnable mapping function is proposed to improve the performance of linear attention. Empirically, the Sub-Adjacent Transformer achieves state-of-the-art performance across six real-world anomaly detection benchmarks, covering diverse fields such as server monitoring, space exploration, and water treatment.