# OLED: One-Class Learned Encoder-Decoder Network with Adversarial Context Masking for Novelty Detection

arXiv: https://arxiv.org/abs/2103.14953 · HTML: https://mlsanity.com/paper/2103.14953/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 58 references that are in the map.
- First posted on arXiv 27 Mar 2021.

## Builds on (most cited first, 30 of 30)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Neural Turing Machines](https://mlsanity.com/paper/1410.5401/) (2014)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [Future Frame Prediction for Anomaly Detection -- A New Baseline](https://mlsanity.com/paper/1712.09867/) (2017)
- [Memory Networks](https://mlsanity.com/paper/1410.3916/) (2014)
- [Deep Learning for Anomaly Detection: A Survey](https://mlsanity.com/paper/1901.03407/) (2019)
- [Multiresolution Knowledge Distillation for Anomaly Detection](https://mlsanity.com/paper/2011.11108/) (2020)
- [Learning Memory-guided Normality for Anomaly Detection](https://mlsanity.com/paper/2003.13228/) (2020)
- [Adversarially Learned One-Class Classifier for Novelty Detection](https://mlsanity.com/paper/1802.09088/) (2018)
- [OCGAN: One-class Novelty Detection Using GANs with Constrained Latent Representations](https://mlsanity.com/paper/1903.08550/) (2019)
- [Latent Space Autoregression for Novelty Detection](https://mlsanity.com/paper/1807.01653/) (2018)
- [Anomaly Detection in Video via Self-Supervised and Multi-Task Learning](https://mlsanity.com/paper/2011.07491/) (2020)
- [Autoencoders for Unsupervised Anomaly Segmentation in Brain MR Images: A Comparative Study](https://mlsanity.com/paper/2004.03271/) (2020)
- [Anomaly Detection in Video Sequence with Appearance-Motion Correspondence](https://mlsanity.com/paper/1908.06351/) (2019)
- [Old is Gold: Redefining the Adversarially Learned One-Class Classifier Training Paradigm](https://mlsanity.com/paper/2004.07657/) (2020)
- [Deep Structured Energy Based Models for Anomaly Detection](https://mlsanity.com/paper/1605.07717/) (2016)
- [Object-centric Auto-encoders and Dummy Anomalies for Abnormal Event Detection in Video](https://mlsanity.com/paper/1812.04960/) (2018)
- [Abnormal Event Detection in Videos using Generative Adversarial Nets](https://mlsanity.com/paper/1708.09644/) (2017)
- [Learning Deep Representations of Appearance and Motion for Anomalous Event Detection](https://mlsanity.com/paper/1510.01553/) (2015)
- [Context-encoding Variational Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1812.05941/) (2018)
- [One-shot Learning with Memory-Augmented Neural Networks](https://mlsanity.com/paper/1605.06065/) (2016)
- [Abnormal Event Detection in Videos using Spatiotemporal Autoencoder](https://mlsanity.com/paper/1701.01546/) (2017)
- [Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledge](https://mlsanity.com/paper/1709.09121/) (2017)
- [Training Adversarial Discriminators for Cross-channel Abnormal Event Detection in Crowds](https://mlsanity.com/paper/1706.07680/) (2017)
- [STAN: Spatio-Temporal Adversarial Networks for Abnormal Event Detection](https://mlsanity.com/paper/1804.08381/) (2018)
- [Memorization Precedes Generation: Learning Unsupervised GANs with Memory Networks](https://mlsanity.com/paper/1803.01500/) (2018)
- [Learning to Generate with Memory](https://mlsanity.com/paper/1602.07416/) (2016)

## Built on by (most cited first, 14 of 14)

- [Generative Cooperative Learning for Unsupervised Video Anomaly Detection](https://mlsanity.com/paper/2203.03962/) (2022)
- [Video Anomaly Detection via Spatio-Temporal Pseudo-Anomaly Generation : A Unified Approach](https://mlsanity.com/paper/2311.16514/) (2023)
- [Hadamard product in deep learning: Introduction, Advances and Challenges](https://mlsanity.com/paper/2504.13112/) (2025)
- [Exploiting Autoencoder's Weakness to Generate Pseudo Anomalies](https://mlsanity.com/paper/2405.05886/) (2024)
- [Robust Semi-Supervised Anomaly Detection via Adversarially Learned Continuous Noise Corruption](https://mlsanity.com/paper/2303.03925/) (2023)
- [Connective Reconstruction-based Novelty Detection](https://mlsanity.com/paper/2210.13917/) (2022)
- [Towards Realistic Out-of-Distribution Detection: A Novel Evaluation Framework for Improving Generalization in OOD Detection](https://mlsanity.com/paper/2211.10892/) (2022)
- [Self-supervised New Activity Detection in Sensor-based Smart Environments](https://mlsanity.com/paper/2401.10288/) (2024)
- [Efficient Slice Anomaly Detection Network for 3D Brain MRI Volume](https://mlsanity.com/paper/2408.15958/) (2024)
- [Anomaly Detection with Adversarially Learned Perturbations of Latent Space](https://mlsanity.com/paper/2207.01106/) (2022)
- [Augment to Detect Anomalies with Continuous Labelling](https://mlsanity.com/paper/2207.01112/) (2022)
- [A Robust Likelihood Model for Novelty Detection](https://mlsanity.com/paper/2306.03331/) (2023)
- [Adversarial Masked Image Inpainting for Robust Detection of Mpox and Non-Mpox](https://mlsanity.com/paper/2310.06318/) (2023)
- [Enhancing Anomaly Detection Generalization through Knowledge Exposure: The Dual Effects of Augmentation](https://mlsanity.com/paper/2406.10617/) (2024)

## Abstract (arXiv)

Novelty detection is the task of recognizing samples that do not belong to the distribution of the target class. During training, the novelty class is absent, preventing the use of traditional classification approaches. Deep autoencoders have been widely used as a base of many unsupervised novelty detection methods. In particular, context autoencoders have been successful in the novelty detection task because of the more effective representations they learn by reconstructing original images from randomly masked images. However, a significant drawback of context autoencoders is that random masking fails to consistently cover important structures of the input image, leading to suboptimal representations - especially for the novelty detection task. In this paper, to optimize input masking, we have designed a framework consisting of two competing networks, a Mask Module and a Reconstructor. The Mask Module is a convolutional autoencoder that learns to generate optimal masks that cover the most important parts of images. Alternatively, the Reconstructor is a convolutional encoder-decoder that aims to reconstruct unperturbed images from masked images. The networks are trained in an adversarial manner in which the Mask Module generates masks that are applied to images given to the Reconstructor. In this way, the Mask Module seeks to maximize the reconstruction error that the Reconstructor is minimizing. When applied to novelty detection, the proposed approach learns semantically richer representations compared to context autoencoders and enhances novelty detection at test time through more optimal masking. Novelty detection experiments on the MNIST and CIFAR-10 image datasets demonstrate the proposed approach's superiority over cutting-edge methods. In a further experiment on the UCSD video dataset for novelty detection, the proposed approach achieves state-of-the-art results.