# Novelty Detection via Contrastive Learning with Negative Data Augmentation

arXiv: https://arxiv.org/abs/2106.09958 · HTML: https://mlsanity.com/paper/2106.09958/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 18 Jun 2021.

## Builds on (most cited first, 20 of 20)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances](https://mlsanity.com/paper/2007.08176/) (2020)
- [Adversarially Learned One-Class Classifier for Novelty Detection](https://mlsanity.com/paper/1802.09088/) (2018)
- [OCGAN: One-class Novelty Detection Using GANs with Constrained Latent Representations](https://mlsanity.com/paper/1903.08550/) (2019)
- [Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervision](https://mlsanity.com/paper/1803.11097/) (2018)
- [Negative Data Augmentation](https://mlsanity.com/paper/2102.05113/) (2021)
- [DOPING: Generative Data Augmentation for Unsupervised Anomaly Detection with GAN](https://mlsanity.com/paper/1808.07632/) (2018)
- [Learning Generalizable and Identity-Discriminative Representations for Face Anti-Spoofing](https://mlsanity.com/paper/1901.05602/) (2019)

## Built on by (most cited first, 10 of 10)

- [Self-Supervised Anomaly Detection in Computer Vision and Beyond: A Survey and Outlook](https://mlsanity.com/paper/2205.05173/) (2022)
- [Does Negative Sampling Matter? A Review with Insights into its Theory and Applications](https://mlsanity.com/paper/2402.17238/) (2024)
- [Contrastive Knowledge Graph Error Detection](https://mlsanity.com/paper/2211.10030/) (2022)
- [PatchGuard: Adversarially Robust Anomaly Detection and Localization through Vision Transformers and Pseudo Anomalies](https://mlsanity.com/paper/2506.09237/) (2025)
- [Unilaterally Aggregated Contrastive Learning with Hierarchical Augmentation for Anomaly Detection](https://mlsanity.com/paper/2308.10155/) (2023)
- [Rethinking Rotation in Self-Supervised Contrastive Learning: Adaptive Positive or Negative Data Augmentation](https://mlsanity.com/paper/2210.12681/) (2022)
- [Removing Geometric Bias in One-Class Anomaly Detection with Adaptive Feature Perturbation](https://mlsanity.com/paper/2503.05520/) (2025)
- [Identity Documents Authentication based on Forgery Detection of Guilloche Pattern](https://mlsanity.com/paper/2206.10989/) (2022)
- [Anomaly Detection via Multi-Scale Contrasted Memory](https://mlsanity.com/paper/2211.09041/) (2022)
- [Empirical Evidences for the Effects of Feature Diversity in Open Set Recognition and Continual Learning](https://mlsanity.com/paper/2508.13005/) (2025)

## Abstract (arXiv)

Novelty detection is the process of determining whether a query example differs from the learned training distribution. Previous methods attempt to learn the representation of the normal samples via generative adversarial networks (GANs). However, they will suffer from instability training, mode dropping, and low discriminative ability. Recently, various pretext tasks (e.g. rotation prediction and clustering) have been proposed for self-supervised learning in novelty detection. However, the learned latent features are still low discriminative. We overcome such problems by introducing a novel decoder-encoder framework. Firstly, a generative network (a.k.a. decoder) learns the representation by mapping the initialized latent vector to an image. In particular, this vector is initialized by considering the entire distribution of training data to avoid the problem of mode-dropping. Secondly, a contrastive network (a.k.a. encoder) aims to ``learn to compare'' through mutual information estimation, which directly helps the generative network to obtain a more discriminative representation by using a negative data augmentation strategy. Extensive experiments show that our model has significant superiority over cutting-edge novelty detectors and achieves new state-of-the-art results on some novelty detection benchmarks, e.g. CIFAR10 and DCASE. Moreover, our model is more stable for training in a non-adversarial manner, compared to other adversarial based novelty detection methods.