# Self-Supervision for Tackling Unsupervised Anomaly Detection: Pitfalls and Opportunities

arXiv: https://arxiv.org/abs/2308.14380 · HTML: https://mlsanity.com/paper/2308.14380/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 63 references that are in the map.
- First posted on arXiv 28 Aug 2023.

## Builds on (most cited first, 29 of 29)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [CutPaste: Self-Supervised Learning for Anomaly Detection and Localization](https://mlsanity.com/paper/2104.04015/) (2021)
- [GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training](https://mlsanity.com/paper/1805.06725/) (2018)
- [Deep Anomaly Detection Using Geometric Transformations](https://mlsanity.com/paper/1805.10917/) (2018)
- [A Cookbook of Self-Supervised Learning](https://mlsanity.com/paper/2304.12210/) (2023)
- [Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methods](https://mlsanity.com/paper/2205.11508/) (2022)
- [Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges](https://mlsanity.com/paper/2107.05847/) (2021)
- [Learning Data Augmentation Strategies for Object Detection](https://mlsanity.com/paper/1906.11172/) (2019)
- [Self-supervised Learning on Graphs: Contrastive, Generative,or Predictive](https://mlsanity.com/paper/2105.07342/) (2021)
- [Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data](https://mlsanity.com/paper/2002.06716/) (2020)
- [Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions](https://mlsanity.com/paper/1903.03088/) (2019)
- [A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?](https://mlsanity.com/paper/2303.11717/) (2023)
- [High Fidelity Visualization of What Your Self-Supervised Representation Knows About](https://mlsanity.com/paper/2112.09164/) (2021)
- [Unsupervised Model Selection for Variational Disentangled Representation Learning](https://mlsanity.com/paper/1905.12614/) (2019)
- [Calibrated One-class Classification for Unsupervised Time Series Anomaly Detection](https://mlsanity.com/paper/2207.12201/) (2022)
- [The SSL Interplay: Augmentations, Inductive Bias, and Generalization](https://mlsanity.com/paper/2302.02774/) (2023)
- [Exposing Outlier Exposure: What Can Be Learned From Few, One, and Zero Outlier Images](https://mlsanity.com/paper/2205.11474/) (2022)
- [How to Evaluate the Quality of Unsupervised Anomaly Detection Algorithms?](https://mlsanity.com/paper/1607.01152/) (2016)
- [Hyperparameter Sensitivity in Deep Outlier Detection: Analysis and a Scalable Hyper-Ensemble Solution](https://mlsanity.com/paper/2206.07647/) (2022)
- [Toward Unsupervised Outlier Model Selection](https://mlsanity.com/paper/2211.01834/) (2022)
- [Data Augmentation is a Hyperparameter: Cherry-picked Self-Supervision for Unsupervised Anomaly Detection is Creating the Illusion of Success](https://mlsanity.com/paper/2208.07734/) (2022)
- [Understanding the Effect of Bias in Deep Anomaly Detection](https://mlsanity.com/paper/2105.07346/) (2021)
- [DSV: An Alignment Validation Loss for Self-supervised Outlier Model Selection](https://mlsanity.com/paper/2307.06534/) (2023)

## Built on by (most cited first, 2 of 2)

- [Deep Transductive Outlier Detection](https://mlsanity.com/paper/2404.03495/) (2024)
- [Self-Supervised Anomaly Detection in the Wild: Favor Joint Embeddings Methods](https://mlsanity.com/paper/2410.04289/) (2024)

## Abstract (arXiv)

Self-supervised learning (SSL) is a growing torrent that has recently transformed machine learning and its many real world applications, by learning on massive amounts of unlabeled data via self-generated supervisory signals. Unsupervised anomaly detection (AD) has also capitalized on SSL, by self-generating pseudo-anomalies through various data augmentation functions or external data exposure. In this vision paper, we first underline the importance of the choice of SSL strategies on AD performance, by presenting evidences and studies from the AD literature. Equipped with the understanding that SSL incurs various hyperparameters (HPs) to carefully tune, we present recent developments on unsupervised model selection and augmentation tuning for SSL-based AD. We then highlight emerging challenges and future opportunities; on designing new pretext tasks and augmentation functions for different data modalities, creating novel model selection solutions for systematically tuning the SSL HPs, as well as on capitalizing on the potential of pretrained foundation models on AD through effective density estimation.