mlsanityData to 14 Oct 2026

Self-Supervision for Tackling Unsupervised Anomaly Detection: Pitfalls and Opportunities

arXiv 2308.14380 ↗First posted 28 Aug 2023cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) is a growing torrent that has recently transformed machine learning and its many real world applications, by learning on massive amounts of unlabeled data via self-generated supervisory signals. Unsupervised anomaly detection (AD) has also capitalized on SSL, by self-generating pseudo-anomalies through various data augmentation functions or external data exposure. In this vision paper, we first underline the importance of the choice of SSL strategies on AD performance, by presenting evidences and studies from the AD literature. Equipped with the understanding that SSL incurs various hyperparameters (HPs) to carefully tune, we present recent developments on unsupervised model selection and augmentation tuning for SSL-based AD. We then highlight emerging challenges and future opportunities; on designing new pretext tasks and augmentation functions for different data modalities, creating novel model selection solutions for systematically tuning the SSL HPs, as well as on capitalizing on the potential of pretrained foundation models on AD through effective density estimation.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 29 of its 63 references that are in the map. First posted on arXiv 28 Aug 2023.

Builds on

29 of 63 references.

References · 29

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  4. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  5. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  6. 2018Deep Anomaly Detection with Outlier Exposurecited by 942
  7. 2020Self-supervised Learning: Generative or Contrastivecited by 694
  8. 2021CutPaste: Self-Supervised Learning for Anomaly Detection and Localizationcited by 531
  9. 2018GANomaly: Semi-Supervised Anomaly Detection via Adversarial Trainingcited by 409
  10. 2018Deep Anomaly Detection Using Geometric Transformationscited by 269
  11. 2023A Cookbook of Self-Supervised Learningcited by 250
  12. 2022Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methodscited by 149
17 more
  1. 2021Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challengescited by 145
  2. 2019Learning Data Augmentation Strategies for Object Detectioncited by 139
  3. 2021Self-supervised Learning on Graphs: Contrastive, Generative,or Predictivecited by 136
  4. 2020Predicting trends in the quality of state-of-the-art neural networks without access to training or testing datacited by 115
  5. 2019Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functionscited by 105
  6. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  7. 2021High Fidelity Visualization of What Your Self-Supervised Representation Knows Aboutcited by 66
  8. 2019Unsupervised Model Selection for Variational Disentangled Representation Learningcited by 46
  9. 2022Calibrated One-class Classification for Unsupervised Time Series Anomaly Detectioncited by 40
  10. 2023The SSL Interplay: Augmentations, Inductive Bias, and Generalizationcited by 40
  11. 2022Exposing Outlier Exposure: What Can Be Learned From Few, One, and Zero Outlier Imagescited by 35
  12. 2016How to Evaluate the Quality of Unsupervised Anomaly Detection Algorithms?cited by 24
  13. 2022Hyperparameter Sensitivity in Deep Outlier Detection: Analysis and a Scalable Hyper-Ensemble Solutioncited by 24
  14. 2022Toward Unsupervised Outlier Model Selectioncited by 19
  15. 2022Data Augmentation is a Hyperparameter: Cherry-picked Self-Supervision for Unsupervised Anomaly Detection is Creating the Illusion of Successcited by 11
  16. 2021Understanding the Effect of Bias in Deep Anomaly Detectioncited by 7
  17. 2023DSV: An Alignment Validation Loss for Self-supervised Outlier Model Selectioncited by 6

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024Deep Transductive Outlier Detectioncited by 1
  2. 2024Self-Supervised Anomaly Detection in the Wild: Favor Joint Embeddings Methodscited by 0