# MAD: Self-Supervised Masked Anomaly Detection Task for Multivariate Time Series

arXiv: https://arxiv.org/abs/2205.02100 · HTML: https://mlsanity.com/paper/2205.02100/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 48 references that are in the map.
- First posted on arXiv 4 May 2022.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [Unsupervised Learning of Video Representations using LSTMs](https://mlsanity.com/paper/1502.04681/) (2015)
- [Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding](https://mlsanity.com/paper/1802.04431/) (2018)
- [Deep Learning for Anomaly Detection: A Survey](https://mlsanity.com/paper/1901.03407/) (2019)
- [FNet: Mixing Tokens with Fourier Transforms](https://mlsanity.com/paper/2105.03824/) (2021)
- [Learning Representations for Automatic Colorization](https://mlsanity.com/paper/1603.06668/) (2016)
- [LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection](https://mlsanity.com/paper/1607.00148/) (2016)
- [Tracking Emerges by Colorizing Videos](https://mlsanity.com/paper/1806.09594/) (2018)
- [Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case](https://mlsanity.com/paper/2001.08317/) (2020)
- [LogBERT: Log Anomaly Detection via BERT](https://mlsanity.com/paper/2103.04475/) (2021)
- [Unsupervised Learning by Predicting Noise](https://mlsanity.com/paper/1704.05310/) (2017)
- [Fault Detection and Identification using Bayesian Recurrent Neural Networks](https://mlsanity.com/paper/1911.04386/) (2019)
- [Self-Supervised Representation Learning for Visual Anomaly Detection](https://mlsanity.com/paper/2006.09654/) (2020)

## Built on by (most cited first, 12 of 12)

- [Self-Supervised Anomaly Detection in Computer Vision and Beyond: A Survey and Outlook](https://mlsanity.com/paper/2205.05173/) (2022)
- [Explainable Time Series Anomaly Detection using Masked Latent Generative Modeling](https://mlsanity.com/paper/2311.12550/) (2023)
- [DyEdgeGAT: Dynamic Edge via Graph Attention for Early Fault Detection in IIoT Systems](https://mlsanity.com/paper/2307.03761/) (2023)
- [Masked Multi-Step Probabilistic Forecasting for Short-to-Mid-Term Electricity Demand](https://mlsanity.com/paper/2302.06818/) (2023)
- [Masked Multi-Step Multivariate Time Series Forecasting with Future Information](https://mlsanity.com/paper/2209.14413/) (2022)
- [NeuCoReClass AD: Redefining Self-Supervised Time Series Anomaly Detection](https://mlsanity.com/paper/2508.00909/) (2025)
- [One Masked Model is All You Need for Sensor Fault Detection, Isolation and Accommodation](https://mlsanity.com/paper/2403.16153/) (2024)
- [End-To-End Self-Tuning Self-Supervised Time Series Anomaly Detection](https://mlsanity.com/paper/2404.02865/) (2024)
- [Hypergraph Learning based Recommender System for Anomaly Detection, Control and Optimization](https://mlsanity.com/paper/2408.11359/) (2024)
- [Pi-transformer: A prior-informed dual-attention model for multivariate time-series anomaly detection](https://mlsanity.com/paper/2509.19985/) (2025)
- [Enhancing ECG Classification Robustness with Lightweight Unsupervised Anomaly Detection Filters](https://mlsanity.com/paper/2510.26501/) (2025)
- [VAN-AD: Visual Masked Autoencoder with Normalizing Flow For Time Series Anomaly Detection](https://mlsanity.com/paper/2603.26842/) (2026)

## Abstract (arXiv)

In this paper, we introduce Masked Anomaly Detection (MAD), a general self-supervised learning task for multivariate time series anomaly detection. With the increasing availability of sensor data from industrial systems, being able to detecting anomalies from streams of multivariate time series data is of significant importance. Given the scarcity of anomalies in real-world applications, the majority of literature has been focusing on modeling normality. The learned normal representations can empower anomaly detection as the model has learned to capture certain key underlying data regularities. A typical formulation is to learn a predictive model, i.e., use a window of time series data to predict future data values. In this paper, we propose an alternative self-supervised learning task. By randomly masking a portion of the inputs and training a model to estimate them using the remaining ones, MAD is an improvement over the traditional left-to-right next step prediction (NSP) task. Our experimental results demonstrate that MAD can achieve better anomaly detection rates over traditional NSP approaches when using exactly the same neural network (NN) base models, and can be modified to run as fast as NSP models during test time on the same hardware, thus making it an ideal upgrade for many existing NSP-based NN anomaly detection models.