# R2-AD2: Detecting Anomalies by Analysing the Raw Gradient

arXiv: https://arxiv.org/abs/2206.10259 · HTML: https://mlsanity.com/paper/2206.10259/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 46 references that are in the map.
- First posted on arXiv 21 Jun 2022.

## Builds on (most cited first, 30 of 31)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [A Unifying Review of Deep and Shallow Anomaly Detection](https://mlsanity.com/paper/2009.11732/) (2020)
- [GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training](https://mlsanity.com/paper/1805.06725/) (2018)
- [Deep Learning for Anomaly Detection: A Survey](https://mlsanity.com/paper/1901.03407/) (2019)
- [On the Importance of Gradients for Detecting Distributional Shifts in the Wild](https://mlsanity.com/paper/2110.00218/) (2021)
- [Deep Semi-Supervised Anomaly Detection](https://mlsanity.com/paper/1906.02694/) (2019)
- [Deep Anomaly Detection Using Geometric Transformations](https://mlsanity.com/paper/1805.10917/) (2018)
- [Classification-Based Anomaly Detection for General Data](https://mlsanity.com/paper/2005.02359/) (2020)
- [MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks](https://mlsanity.com/paper/1901.04997/) (2019)
- [A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges](https://mlsanity.com/paper/2110.14051/) (2021)
- [Deep Anomaly Detection with Deviation Networks](https://mlsanity.com/paper/1911.08623/) (2019)
- [Learning and Evaluating Representations for Deep One-class Classification](https://mlsanity.com/paper/2011.02578/) (2020)
- [DROCC: Deep Robust One-Class Classification](https://mlsanity.com/paper/2002.12718/) (2020)
- [Anomaly Detection using One-Class Neural Networks](https://mlsanity.com/paper/1802.06360/) (2018)
- [Learning Representations of Ultrahigh-dimensional Data for Random Distance-based Outlier Detection](https://mlsanity.com/paper/1806.04808/) (2018)
- [Backpropagated Gradient Representations for Anomaly Detection](https://mlsanity.com/paper/2007.09507/) (2020)
- [Robust Anomaly Detection in Images using Adversarial Autoencoders](https://mlsanity.com/paper/1901.06355/) (2019)
- [Anomaly Detection using Autoencoders in High Performance Computing Systems](https://mlsanity.com/paper/1811.05269/) (2018)
- [GraN: An Efficient Gradient-Norm Based Detector for Adversarial and Misclassified Examples](https://mlsanity.com/paper/2004.09179/) (2020)
- [Autoencoding Binary Classifiers for Supervised Anomaly Detection](https://mlsanity.com/paper/1903.10709/) (2019)
- [Novelty Detection Through Model-Based Characterization of Neural Networks](https://mlsanity.com/paper/2008.06094/) (2020)
- [Anomaly Detection with Adversarial Dual Autoencoders](https://mlsanity.com/paper/1902.06924/) (2019)
- [Gradient-Based Adversarial and Out-of-Distribution Detection](https://mlsanity.com/paper/2206.08255/) (2022)
- [Gradient-based Novelty Detection Boosted by Self-supervised Binary Classification](https://mlsanity.com/paper/2112.09815/) (2021)
- [ESAD: End-to-end Deep Semi-supervised Anomaly Detection](https://mlsanity.com/paper/2012.04905/) (2020)
- [Understanding the Effect of Bias in Deep Anomaly Detection](https://mlsanity.com/paper/2105.07346/) (2021)
- [Open-Set Recognition with Gradient-Based Representations](https://mlsanity.com/paper/2206.08229/) (2022)

## Built on by (most cited first, 2 of 2)

- [Semisupervised Anomaly Detection using Support Vector Regression with Quantum Kernel](https://mlsanity.com/paper/2308.00583/) (2023)
- [QUACK: Quantum Aligned Centroid Kernel](https://mlsanity.com/paper/2405.00304/) (2024)

## Abstract (arXiv)

Neural networks follow a gradient-based learning scheme, adapting their mapping parameters by back-propagating the output loss. Samples unlike the ones seen during training cause a different gradient distribution. Based on this intuition, we design a novel semi-supervised anomaly detection method called R2-AD2. By analysing the temporal distribution of the gradient over multiple training steps, we reliably detect point anomalies in strict semi-supervised settings. Instead of domain dependent features, we input the raw gradient caused by the sample under test to an end-to-end recurrent neural network architecture. R2-AD2 works in a purely data-driven way, thus is readily applicable in a variety of important use cases of anomaly detection.