# Graph Mixture of Experts and Memory-augmented Routers for Multivariate Time Series Anomaly Detection

arXiv: https://arxiv.org/abs/2412.19108 · HTML: https://mlsanity.com/paper/2412.19108/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in MoE for time series and graphs, one of 12 directions in Mixture-of-experts.
- Builds on 27 of its 52 references that are in the map.
- First posted on arXiv 26 Dec 2024.

## Placed in

- Mixture-of-experts / [MoE for time series and graphs](https://mlsanity.com/moe/moe-for-time-series-and-graphs/)

## Builds on (most cited first, 27 of 27)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://mlsanity.com/paper/2006.16668/) (2020)
- [Representation Learning on Graphs with Jumping Knowledge Networks](https://mlsanity.com/paper/1806.03536/) (2018)
- [Graph WaveNet for Deep Spatial-Temporal Graph Modeling](https://mlsanity.com/paper/1906.00121/) (2019)
- [Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks](https://mlsanity.com/paper/2005.11650/) (2020)
- [Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding](https://mlsanity.com/paper/1802.04431/) (2018)
- [Generating Radiology Reports via Memory-driven Transformer](https://mlsanity.com/paper/2010.16056/) (2020)
- [CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows](https://mlsanity.com/paper/2107.12571/) (2021)
- [BASE Layers: Simplifying Training of Large, Sparse Models](https://mlsanity.com/paper/2103.16716/) (2021)
- [Graph Neural Network-Based Anomaly Detection in Multivariate Time Series](https://mlsanity.com/paper/2106.06947/) (2021)
- [Deep Semi-Supervised Anomaly Detection](https://mlsanity.com/paper/1906.02694/) (2019)
- [Multivariate Time-series Anomaly Detection via Graph Attention Network](https://mlsanity.com/paper/2009.02040/) (2020)
- [Time-Series Anomaly Detection Service at Microsoft](https://mlsanity.com/paper/1906.03821/) (2019)
- [Multivariate Probabilistic Time Series Forecasting via Conditioned Normalizing Flows](https://mlsanity.com/paper/2002.06103/) (2020)
- [DROCC: Deep Robust One-Class Classification](https://mlsanity.com/paper/2002.12718/) (2020)
- [Graph Mixture of Experts: Learning on Large-Scale Graphs with Explicit Diversity Modeling](https://mlsanity.com/paper/2304.02806/) (2023)
- [Structural Temporal Graph Neural Networks for Anomaly Detection in Dynamic Graphs](https://mlsanity.com/paper/2005.07427/) (2020)
- [Memory Augmented Graph Neural Networks for Sequential Recommendation](https://mlsanity.com/paper/1912.11730/) (2019)
- [ADA-GAD: Anomaly-Denoised Autoencoders for Graph Anomaly Detection](https://mlsanity.com/paper/2312.14535/) (2023)
- [Detecting Multivariate Time Series Anomalies with Zero Known Label](https://mlsanity.com/paper/2208.02108/) (2022)
- [Dynamic Memory based Attention Network for Sequential Recommendation](https://mlsanity.com/paper/2102.09269/) (2021)
- [A Tale of HodgeRank and Spectral Method: Target Attack Against Rank Aggregation Is the Fixed Point of Adversarial Game](https://mlsanity.com/paper/2209.05742/) (2022)
- [Poisoning Attack against Estimating from Pairwise Comparisons](https://mlsanity.com/paper/2107.01854/) (2021)
- [Neural Relational Inference with Efficient Message Passing Mechanisms](https://mlsanity.com/paper/2101.09486/) (2021)
- [Sequential Manipulation Against Rank Aggregation: Theory and Algorithm](https://mlsanity.com/paper/2407.01916/) (2024)
- [Heterogeneous Interaction Modeling With Reduced Accumulated Error for Multi-Agent Trajectory Prediction](https://mlsanity.com/paper/2410.21342/) (2024)

## Built on by (most cited first, 20 of 20)

- [Can Multimodal LLMs Perform Time Series Anomaly Detection?](https://mlsanity.com/paper/2502.17812/) (2025)
- [Learning Soft Sparse Shapes for Efficient Time-Series Classification](https://mlsanity.com/paper/2505.06892/) (2025)
- [EmoVerse: A MLLMs-Driven Emotion Representation Dataset for Interpretable Visual Emotion Analysis](https://mlsanity.com/paper/2511.12554/) (2025)
- [Towards Accurate Emotion-Attributed Video Captioning via Fine-grained Emotion-Cause Pair Extraction](https://mlsanity.com/paper/2606.08566/) (2026)
- [Audio-Visual Exchange-Aware Token Pruning for Efficient Audio-Visual Captioning](https://mlsanity.com/paper/2606.10533/) (2026)
- [EmoStyle: Affective Conditioning of Style-Specialist Experts for Emotional Image Generation](https://mlsanity.com/paper/2607.10165/) (2026)
- [Mixture-of-Experts for Personalized and Semantic-Aware Next Location Prediction](https://mlsanity.com/paper/2505.24597/) (2025)
- [Variational Mixture of Graph Neural Experts for Alzheimer's Disease Recognition across Frequency Bands in EEG Brain Networks](https://mlsanity.com/paper/2510.11917/) (2025)
- [Cluster-Aware Causal Mixer for Online Anomaly Detection in Multivariate Time Series](https://mlsanity.com/paper/2506.00188/) (2025)
- [Modality as Heterogeneity: Node Splitting and Graph Rewiring for Multimodal Graph Learning](https://mlsanity.com/paper/2602.00067/) (2026)
- [Geometry-aware Gaussian Prior and Axial Attention for Cervical Cytology Image Classification](https://mlsanity.com/paper/2607.10278/) (2026)
- [Dynamic Adaptive Parsing of Temporal and Cross-Variable Patterns for Network State Classification](https://mlsanity.com/paper/2509.11601/) (2025)
- [Explainable Heterogeneous Anomaly Detection in Financial Networks via Adaptive Expert Routing](https://mlsanity.com/paper/2510.17088/) (2025)
- [Routing-Aware Explanations for Mixture of Experts Graph Models in Malware Detection](https://mlsanity.com/paper/2602.19025/) (2026)
- [Affinity Is Not Enough: Recovering the Free Energy Principle in Mixture-of-Experts](https://mlsanity.com/paper/2605.00604/) (2026)
- [Rationale-Guided Knowledge Distillation for Cross-Lingual Stance Detection](https://mlsanity.com/paper/2607.18693/) (2026)
- [DSPO: Diversity-aware Subjective Policy Optimization for Robust Emotional Reasoning](https://mlsanity.com/paper/2609.36775/) (2026)
- [Causal-EVC: Breaking Emotional Spurious Causality via Spatiotemporal Grounding and Counterfactual Intervention](https://mlsanity.com/paper/2609.36776/) (2026)
- [Decoding Affective Nuances: Enhancing MLLMs via Hierarchical Emotion Reasoning and Contrastive Discriminative Pruning](https://mlsanity.com/paper/2609.36782/) (2026)
- [FreSia: Frequency-Semantic Instantiation and Alignment for Multivariate Time Series Analysis](https://mlsanity.com/paper/2610.05726/) (2026)

## Abstract (arXiv)

Multivariate time series (MTS) anomaly detection is a critical task that involves identifying abnormal patterns or events in data that consist of multiple interrelated time series. In order to better model the complex interdependence between entities and the various inherent characteristics of each entity, the GNN based methods are widely adopted by existing methods. In each layer of GNN, node features aggregate information from their neighboring nodes to update their information. In doing so, from shallow layer to deep layer in GNN, original individual node features continue to be weakened and more structural information,i.e., from short-distance neighborhood to long-distance neighborhood, continues to be enhanced. However, research to date has largely ignored the understanding of how hierarchical graph information is represented and their characteristics that can benefit anomaly detection. Existing methods simply leverage the output from the last layer of GNN for anomaly estimation while neglecting the essential information contained in the intermediate GNN layers. To address such limitations, in this paper, we propose a Graph Mixture of Experts (Graph-MoE) network for multivariate time series anomaly detection, which incorporates the mixture of experts (MoE) module to adaptively represent and integrate hierarchical multi-layer graph information into entity representations. It is worth noting that our Graph-MoE can be integrated into any GNN-based MTS anomaly detection method in a plug-and-play manner. In addition, the memory-augmented routers are proposed in this paper to capture the correlation temporal information in terms of the global historical features of MTS to adaptively weigh the obtained entity representations to achieve successful anomaly estimation. Extensive experiments on five challenging datasets prove the superiority of our approach and each proposed module.