# GADformer: A Transparent Transformer Model for Group Anomaly Detection on Trajectories

arXiv: https://arxiv.org/abs/2303.09841 · HTML: https://mlsanity.com/paper/2303.09841/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 27 references that are in the map.
- First posted on arXiv 17 Mar 2023.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [Deep Learning for Anomaly Detection: A Review](https://mlsanity.com/paper/2007.02500/) (2020)
- [Anomaly Detection with Density Estimation](https://mlsanity.com/paper/2001.04990/) (2020)
- [On the Nature and Types of Anomalies: A Review of Deviations in Data](https://mlsanity.com/paper/2007.15634/) (2020)
- [Mutual Distillation Learning Network for Trajectory-User Linking](https://mlsanity.com/paper/2205.03773/) (2022)
- [One-Class Support Measure Machines for Group Anomaly Detection](https://mlsanity.com/paper/1303.0309/) (2013)
- [GLAD: Group Anomaly Detection in Social Media Analysis- Extended Abstract](https://mlsanity.com/paper/1410.1940/) (2014)
- [Anomaly Detection in Trajectory Data with Normalizing Flows](https://mlsanity.com/paper/2004.05958/) (2020)
- [Group Anomaly Detection using Deep Generative Models](https://mlsanity.com/paper/1804.04876/) (2018)
- [Do Transformer Attention Heads Provide Transparency in Abstractive Summarization?](https://mlsanity.com/paper/1907.00570/) (2019)

## Built on by (most cited first, 1 of 1)

- [TrajMind: Chaining Role-Specialized LoRAs for Fast-and-Slow Collective Trajectory Anomaly Diagnosis](https://mlsanity.com/paper/2609.02540/) (2026)

## Abstract (arXiv)

Group Anomaly Detection (GAD) identifies unusual pattern in groups where individual members might not be anomalous. This task is of major importance across multiple disciplines, in which also sequences like trajectories can be considered as a group. As groups become more diverse in heterogeneity and size, detecting group anomalies becomes challenging, especially without supervision. Though Recurrent Neural Networks are well established deep sequence models, their performance can decrease with increasing sequence lengths. Hence, this paper introduces GADformer, a BERT-based model for attention-driven GAD on trajectories in unsupervised and semi-supervised settings. We demonstrate how group anomalies can be detected by attention-based GAD. We also introduce the Block-Attention-anomaly-Score (BAS) to enhance model transparency by scoring attention patterns. In addition to that, synthetic trajectory generation allows various ablation studies. In extensive experiments we investigate our approach versus related works in their robustness for trajectory noise and novelties on synthetic data and three real world datasets.