# Attention and Autoencoder Hybrid Model for Unsupervised Online Anomaly Detection

arXiv: https://arxiv.org/abs/2401.03322 · HTML: https://mlsanity.com/paper/2401.03322/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 6 Jan 2024.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [Learning Temporal Regularity in Video Sequences](https://mlsanity.com/paper/1604.04574/) (2016)
- [Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case](https://mlsanity.com/paper/2001.08317/) (2020)
- [Anomaly Detection in Univariate Time-series: A Survey on the State-of-the-Art](https://mlsanity.com/paper/2004.00433/) (2020)
- [Deep Learning-Based Anomaly Detection in Cyber-Physical Systems: Progress and Opportunities](https://mlsanity.com/paper/2003.13213/) (2020)
- [Anomaly Detection for a Water Treatment System Using Unsupervised Machine Learning](https://mlsanity.com/paper/1709.05342/) (2017)
- [Real-Time Sensor Anomaly Detection and Recovery in Connected Automated Vehicle Sensors](https://mlsanity.com/paper/1911.01531/) (2019)
- [Memory-free Online Change-point Detection: A Novel Neural Network Approach](https://mlsanity.com/paper/2207.03932/) (2022)

## Built on by (most cited first, 1 of 1)

- [Community-Based Model Sharing and Generalisation: Anomaly Detection in IoT Temperature Sensor Networks](https://mlsanity.com/paper/2601.05984/) (2026)

## Abstract (arXiv)

This paper introduces a hybrid attention and autoencoder (AE) model for unsupervised online anomaly detection in time series. The autoencoder captures local structural patterns in short embeddings, while the attention model learns long-term features, facilitating parallel computing with positional encoding. Unique in its approach, our proposed hybrid model combines attention and autoencoder for the first time in time series anomaly detection. It employs an attention-based mechanism, akin to the deep transformer model, with key architectural modifications for predicting the next time step window in the autoencoder's latent space. The model utilizes a threshold from the validation dataset for anomaly detection and introduces an alternative method based on analyzing the first statistical moment of error, improving accuracy without dependence on a validation dataset. Evaluation on diverse real-world benchmark datasets and comparing with other well-established models, confirms the effectiveness of our proposed model in anomaly detection.