# End-To-End Self-Tuning Self-Supervised Time Series Anomaly Detection

arXiv: https://arxiv.org/abs/2404.02865 · HTML: https://mlsanity.com/paper/2404.02865/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 54 references that are in the map.
- First posted on arXiv 3 Apr 2024.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [CutPaste: Self-Supervised Learning for Anomaly Detection and Localization](https://mlsanity.com/paper/2104.04015/) (2021)
- [CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances](https://mlsanity.com/paper/2007.08176/) (2020)
- [Deep Anomaly Detection Using Geometric Transformations](https://mlsanity.com/paper/1805.10917/) (2018)
- [Time Series Data Augmentation for Deep Learning: A Survey](https://mlsanity.com/paper/2002.12478/) (2020)
- [LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection](https://mlsanity.com/paper/1607.00148/) (2016)
- [A review on outlier/anomaly detection in time series data](https://mlsanity.com/paper/2002.04236/) (2020)
- [Time-Series Anomaly Detection Service at Microsoft](https://mlsanity.com/paper/1906.03821/) (2019)
- [TimeGPT-1](https://mlsanity.com/paper/2310.03589/) (2023)
- [Neural Transformation Learning for Deep Anomaly Detection Beyond Images](https://mlsanity.com/paper/2103.16440/) (2021)
- [Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects](https://mlsanity.com/paper/2306.10125/) (2023)
- [RobustTAD: Robust Time Series Anomaly Detection via Decomposition and Convolutional Neural Networks](https://mlsanity.com/paper/2002.09545/) (2020)
- [Calibrated One-class Classification for Unsupervised Time Series Anomaly Detection](https://mlsanity.com/paper/2207.12201/) (2022)
- [Masked Contrastive Learning for Anomaly Detection](https://mlsanity.com/paper/2105.08793/) (2021)
- [Toward Unsupervised Outlier Model Selection](https://mlsanity.com/paper/2211.01834/) (2022)
- [Adaptive Memory Networks with Self-supervised Learning for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2201.00464/) (2022)
- [MAD: Self-Supervised Masked Anomaly Detection Task for Multivariate Time Series](https://mlsanity.com/paper/2205.02100/) (2022)
- [Data Augmentation is a Hyperparameter: Cherry-picked Self-Supervision for Unsupervised Anomaly Detection is Creating the Illusion of Success](https://mlsanity.com/paper/2208.07734/) (2022)
- [DSV: An Alignment Validation Loss for Self-supervised Outlier Model Selection](https://mlsanity.com/paper/2307.06534/) (2023)
- [Fast Unsupervised Deep Outlier Model Selection with Hypernetworks](https://mlsanity.com/paper/2307.10529/) (2023)
- [Self-Tuning Self-Supervised Image Anomaly Detection](https://mlsanity.com/paper/2306.12033/) (2023)

## Abstract (arXiv)

Time series anomaly detection (TSAD) finds many applications such as monitoring environmental sensors, industry KPIs, patient biomarkers, etc. A two-fold challenge for TSAD is a versatile and unsupervised model that can detect various different types of time series anomalies (spikes, discontinuities, trend shifts, etc.) without any labeled data. Modern neural networks have outstanding ability in modeling complex time series. Self-supervised models in particular tackle unsupervised TSAD by transforming the input via various augmentations to create pseudo anomalies for training. However, their performance is sensitive to the choice of augmentation, which is hard to choose in practice, while there exists no effort in the literature on data augmentation tuning for TSAD without labels. Our work aims to fill this gap. We introduce TSAP for TSA "on autoPilot", which can (self-)tune augmentation hyperparameters end-to-end. It stands on two key components: a differentiable augmentation architecture and an unsupervised validation loss to effectively assess the alignment between augmentation type and anomaly type. Case studies show TSAP's ability to effectively select the (discrete) augmentation type and associated (continuous) hyperparameters. In turn, it outperforms established baselines, including SOTA self-supervised models, on diverse TSAD tasks exhibiting different anomaly types.