# Learning Graph Structures with Transformer for Multivariate Time Series Anomaly Detection in IoT

arXiv: https://arxiv.org/abs/2104.03466 · HTML: https://mlsanity.com/paper/2104.03466/

## Key facts

- Cited by 104 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 57 references that are in the map.
- First posted on arXiv 8 Apr 2021.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks](https://mlsanity.com/paper/2005.11650/) (2020)
- [Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding](https://mlsanity.com/paper/1802.04431/) (2018)
- [Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting](https://mlsanity.com/paper/2103.07719/) (2021)
- [LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection](https://mlsanity.com/paper/1607.00148/) (2016)
- [Pay Less Attention with Lightweight and Dynamic Convolutions](https://mlsanity.com/paper/1901.10430/) (2019)
- [A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencoder](https://mlsanity.com/paper/1711.00614/) (2017)
- [Synthesizer: Rethinking Self-Attention in Transformer Models](https://mlsanity.com/paper/2005.00743/) (2020)
- [A review on outlier/anomaly detection in time series data](https://mlsanity.com/paper/2002.04236/) (2020)
- [MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks](https://mlsanity.com/paper/1901.04997/) (2019)
- [SpotTune: Transfer Learning through Adaptive Fine-tuning](https://mlsanity.com/paper/1811.08737/) (2018)
- [Multivariate Time-series Anomaly Detection via Graph Attention Network](https://mlsanity.com/paper/2009.02040/) (2020)
- [A* Sampling](https://mlsanity.com/paper/1411.0030/) (2014)
- [The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives](https://mlsanity.com/paper/1909.01380/) (2019)
- [Lite Transformer with Long-Short Range Attention](https://mlsanity.com/paper/2004.11886/) (2020)
- [Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learning](https://mlsanity.com/paper/1711.01239/) (2017)
- [Kitsune: An Ensemble of Autoencoders for Online Network Intrusion Detection](https://mlsanity.com/paper/1802.09089/) (2018)
- [Deep Learning for IoT Big Data and Streaming Analytics: A Survey](https://mlsanity.com/paper/1712.04301/) (2017)
- [Anomaly Detection with Generative Adversarial Networks for Multivariate Time Series](https://mlsanity.com/paper/1809.04758/) (2018)
- [Fixed Encoder Self-Attention Patterns in Transformer-Based Machine Translation](https://mlsanity.com/paper/2002.10260/) (2020)

## Built on by (most cited first, 30 of 104)

- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy](https://mlsanity.com/paper/2110.02642/) (2021)
- [A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection](https://mlsanity.com/paper/2307.03759/) (2023)
- [Deep Learning for Time Series Anomaly Detection: A Survey](https://mlsanity.com/paper/2211.05244/) (2022)
- [DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection](https://mlsanity.com/paper/2306.10347/) (2023)
- [Towards a Rigorous Evaluation of Time-series Anomaly Detection](https://mlsanity.com/paper/2109.05257/) (2021)
- [A Comprehensive Survey on Deep Graph Representation Learning](https://mlsanity.com/paper/2304.05055/) (2023)
- [A Survey on Deep Learning based Time Series Analysis with Frequency Transformation](https://mlsanity.com/paper/2302.02173/) (2023)
- [Towards a General Time Series Anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoders](https://mlsanity.com/paper/2405.15273/) (2024)
- [Interactive AI with Retrieval-Augmented Generation for Next Generation Networking](https://mlsanity.com/paper/2401.11391/) (2024)
- [Graph Transformers: A Survey](https://mlsanity.com/paper/2407.09777/) (2024)
- [Navigating the Metric Maze: A Taxonomy of Evaluation Metrics for Anomaly Detection in Time Series](https://mlsanity.com/paper/2303.01272/) (2023)
- [Deep Learning-based Anomaly Detection and Log Analysis for Computer Networks](https://mlsanity.com/paper/2407.05639/) (2024)
- [Foundation Models for Time Series: A Survey](https://mlsanity.com/paper/2504.04011/) (2025)
- [A Survey of Graph-based Deep Learning for Anomaly Detection in Distributed Systems](https://mlsanity.com/paper/2206.04149/) (2022)
- [Anomaly Detection in Dynamic Graphs: A Comprehensive Survey](https://mlsanity.com/paper/2406.00134/) (2024)
- [MultiRC: Joint Learning for Time Series Anomaly Prediction and Detection with Multi-scale Reconstructive Contrast](https://mlsanity.com/paper/2410.15997/) (2024)
- [Graph Anomaly Detection in Time Series: A Survey](https://mlsanity.com/paper/2302.00058/) (2023)
- [Nominality Score Conditioned Time Series Anomaly Detection by Point/Sequential Reconstruction](https://mlsanity.com/paper/2310.15416/) (2023)
- [Position: Quo Vadis, Unsupervised Time Series Anomaly Detection?](https://mlsanity.com/paper/2405.02678/) (2024)
- [Deep Learning Advancements in Anomaly Detection: A Comprehensive Survey](https://mlsanity.com/paper/2503.13195/) (2025)
- [TiSAT: Time Series Anomaly Transformer](https://mlsanity.com/paper/2203.05167/) (2022)
- [Agentic Retrieval-Augmented Generation for Time Series Analysis](https://mlsanity.com/paper/2408.14484/) (2024)
- [TAMO: Fine-Grained Root Cause Analysis via Tool-Assisted LLM Agent with Multi-Modality Observation Data in Cloud-Native Systems](https://mlsanity.com/paper/2504.20462/) (2025)
- [Federated Variational Learning for Anomaly Detection in Multivariate Time Series](https://mlsanity.com/paper/2108.08404/) (2021)
- [PatchAD: A Lightweight Patch-based MLP-Mixer for Time Series Anomaly Detection](https://mlsanity.com/paper/2401.09793/) (2024)
- [GCAD: Anomaly Detection in Multivariate Time Series from the Perspective of Granger Causality](https://mlsanity.com/paper/2501.13493/) (2025)
- [Unsupervised Deep Learning for IoT Time Series](https://mlsanity.com/paper/2302.03284/) (2023)
- [DyEdgeGAT: Dynamic Edge via Graph Attention for Early Fault Detection in IIoT Systems](https://mlsanity.com/paper/2307.03761/) (2023)
- [DDMT: Denoising Diffusion Mask Transformer Models for Multivariate Time Series Anomaly Detection](https://mlsanity.com/paper/2310.08800/) (2023)

## Abstract (arXiv)

Many real-world IoT systems, which include a variety of internet-connected sensory devices, produce substantial amounts of multivariate time series data. Meanwhile, vital IoT infrastructures like smart power grids and water distribution networks are frequently targeted by cyber-attacks, making anomaly detection an important study topic. Modeling such relatedness is, nevertheless, unavoidable for any efficient and effective anomaly detection system, given the intricate topological and nonlinear connections that are originally unknown among sensors. Furthermore, detecting anomalies in multivariate time series is difficult due to their temporal dependency and stochasticity. This paper presented GTA, a new framework for multivariate time series anomaly detection that involves automatically learning a graph structure, graph convolution, and modeling temporal dependency using a Transformer-based architecture. The connection learning policy, which is based on the Gumbel-softmax sampling approach to learn bi-directed links among sensors directly, is at the heart of learning graph structure. To describe the anomaly information flow between network nodes, we introduced a new graph convolution called Influence Propagation convolution. In addition, to tackle the quadratic complexity barrier, we suggested a multi-branch attention mechanism to replace the original multi-head self-attention method. Extensive experiments on four publicly available anomaly detection benchmarks further demonstrate the superiority of our approach over alternative state-of-the-arts. Codes are available at https://github.com/ZEKAICHEN/GTA.