mlsanityData to 14 Oct 2026

Data Augmentation is a Hyperparameter: Cherry-picked Self-Supervision for Unsupervised Anomaly Detection is Creating the Illusion of Success

arXiv 2208.07734 ↗First posted 16 Aug 2022cs.LG cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) has emerged as a promising alternative to create supervisory signals to real-world problems, avoiding the extensive cost of manual labeling. SSL is particularly attractive for unsupervised tasks such as anomaly detection (AD), where labeled anomalies are rare or often nonexistent. A large catalog of augmentation functions has been used for SSL-based AD (SSAD) on image data, and recent works have reported that the type of augmentation has a significant impact on accuracy. Motivated by those, this work sets out to put image-based SSAD under a larger lens and investigate the role of data augmentation in SSAD. Through extensive experiments on 3 different detector models and across 420 AD tasks, we provide comprehensive numerical and visual evidences that the alignment between data augmentation and anomaly-generating mechanism is the key to the success of SSAD, and in the lack thereof, SSL may even impair accuracy. To the best of our knowledge, this is the first meta-analysis on the role of data augmentation in SSAD.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 40 of its 70 references that are in the map. First posted on arXiv 16 Aug 2022.

Builds on

40 of 70 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  9. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  10. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  11. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
  12. 2017Random Erasing Data Augmentationcited by 1,239
28 more
  1. 2018Deep Anomaly Detection with Outlier Exposurecited by 942
  2. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  3. 2020Self-supervised Learning: Generative or Contrastivecited by 694
  4. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 569
  5. 2021CutPaste: Self-Supervised Learning for Anomaly Detection and Localizationcited by 531
  6. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  7. 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
  8. 2022DeiT III: Revenge of the ViTcited by 419
  9. 2018GANomaly: Semi-Supervised Anomaly Detection via Adversarial Trainingcited by 409
  10. 2015A note on the evaluation of generative modelscited by 292
  11. 2019Deep Semi-Supervised Anomaly Detectioncited by 291
  12. 2018Deep Anomaly Detection Using Geometric Transformationscited by 269
  13. 2021SSD: A Unified Framework for Self-Supervised Outlier Detectioncited by 264
  14. 2020Classification-Based Anomaly Detection for General Datacited by 211
  15. 2021Self-supervised Pretraining of Visual Features in the Wildcited by 198
  16. 2021Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challengescited by 145
  17. 2019Learning Data Augmentation Strategies for Object Detectioncited by 139
  18. 2021Fully Convolutional Cross-Scale-Flows for Image-based Defect Detectioncited by 117
  19. 2018Adversarially Learned Anomaly Detectioncited by 115
  20. 2019Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functionscited by 105
  21. 2022Catching Both Gray and Black Swans: Open-set Supervised Anomaly Detectioncited by 104
  22. 2021Neural Transformation Learning for Deep Anomaly Detection Beyond Imagescited by 99
  23. 2020Rethinking pooling in graph neural networkscited by 73
  24. 2020Rethinking Assumptions in Deep Anomaly Detectioncited by 45
  25. 2022Exposing Outlier Exposure: What Can Be Learned From Few, One, and Zero Outlier Imagescited by 35
  26. 2022Hyperparameter Sensitivity in Deep Outlier Detection: Analysis and a Scalable Hyper-Ensemble Solutioncited by 24
  27. 2021Why Do Self-Supervised Models Transfer? Investigating the Impact of Invariance on Downstream Taskscited by 15
  28. 2021Understanding the Effect of Bias in Deep Anomaly Detectioncited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 2; 2026: 4.

11 papers.

  1. 2024AD-LLM: Benchmarking Large Language Models for Anomaly Detectioncited by 26
  2. 2026From Zero to Hero: Advancing Zero-Shot Foundation Models for Tabular Outlier Detectioncited by 9
  3. 2024On the Detection of Reviewer-Author Collusion Rings From Paper Biddingcited by 7
  4. 2025Robust and Explainable Detector of Time Series Anomaly via Augmenting Multiclass Pseudo-Anomaliescited by 7
  5. 2026MacrOData: New Benchmarks of Thousands of Datasets for Tabular Outlier Detectioncited by 5
  6. 2023Self-Tuning Self-Supervised Image Anomaly Detectioncited by 4
  7. 2026VIP-COP: Context Optimization for Tabular Foundation Modelscited by 3
  8. 2023Self-Supervision for Tackling Unsupervised Anomaly Detection: Pitfalls and Opportunitiescited by 2
  9. 2025NeuCoReClass AD: Redefining Self-Supervised Time Series Anomaly Detectioncited by 1
  10. 2024End-To-End Self-Tuning Self-Supervised Time Series Anomaly Detectioncited by 0
  11. 2026ProtoX-AD: Self-Explainable Time Series Anomaly Detection and Characterizationcited by 0