# Data augmentation and pre-trained networks for extremely low data regimes unsupervised visual inspection

arXiv: https://arxiv.org/abs/2106.01277 · HTML: https://mlsanity.com/paper/2106.01277/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 40 references that are in the map.
- First posted on arXiv 2 Jun 2021.

## Builds on (most cited first, 28 of 28)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization](https://mlsanity.com/paper/2011.08785/) (2020)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances](https://mlsanity.com/paper/2007.08176/) (2020)
- [Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings](https://mlsanity.com/paper/1911.02357/) (2019)
- [Sub-Image Anomaly Detection with Deep Pyramid Correspondences](https://mlsanity.com/paper/2005.02357/) (2020)
- [Deep Semi-Supervised Anomaly Detection](https://mlsanity.com/paper/1906.02694/) (2019)
- [Multiresolution Knowledge Distillation for Anomaly Detection](https://mlsanity.com/paper/2011.11108/) (2020)
- [Deep Anomaly Detection Using Geometric Transformations](https://mlsanity.com/paper/1805.10917/) (2018)
- [Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders](https://mlsanity.com/paper/1807.02011/) (2018)
- [Classification-Based Anomaly Detection for General Data](https://mlsanity.com/paper/2005.02359/) (2020)
- [Autoencoders for Unsupervised Anomaly Segmentation in Brain MR Images: A Comparative Study](https://mlsanity.com/paper/2004.03271/) (2020)
- [Explainable Deep One-Class Classification](https://mlsanity.com/paper/2007.01760/) (2020)
- [Modeling the Distribution of Normal Data in Pre-Trained Deep Features for Anomaly Detection](https://mlsanity.com/paper/2005.14140/) (2020)
- [Deep Nearest Neighbor Anomaly Detection](https://mlsanity.com/paper/2002.10445/) (2020)
- [Learning Deep Features for One-Class Classification](https://mlsanity.com/paper/1801.05365/) (2018)
- [DFR: Deep Feature Reconstruction for Unsupervised Anomaly Segmentation](https://mlsanity.com/paper/2012.07122/) (2020)
- [Rethinking Assumptions in Deep Anomaly Detection](https://mlsanity.com/paper/2006.00339/) (2020)
- [Why is the Mahalanobis Distance Effective for Anomaly Detection?](https://mlsanity.com/paper/2003.00402/) (2020)
- [Novelty Detection Via Blurring](https://mlsanity.com/paper/1911.11943/) (2019)
- [Synthetic training data generation for deep learning based quality inspection](https://mlsanity.com/paper/2104.02980/) (2021)
- [End-to-End Defect Detection in Automated Fiber Placement Based on Artificially Generated Data](https://mlsanity.com/paper/1910.04997/) (2019)
- [Active learning using weakly supervised signals for quality inspection](https://mlsanity.com/paper/2104.02973/) (2021)
- [Task-agnostic Out-of-Distribution Detection Using Kernel Density Estimation](https://mlsanity.com/paper/2006.10712/) (2020)

## Built on by (most cited first, 3 of 3)

- [Optimizing PatchCore for Few/many-shot Anomaly Detection](https://mlsanity.com/paper/2307.10792/) (2023)
- [Data refinement for fully unsupervised visual inspection using pre-trained networks](https://mlsanity.com/paper/2202.12759/) (2022)
- [Deep Learning for Iris Recognition: A Review](https://mlsanity.com/paper/2303.08514/) (2023)

## Abstract (arXiv)

The use of deep features coming from pre-trained neural networks for unsupervised anomaly detection purposes has recently gathered momentum in the computer vision field. In particular, industrial inspection applications can take advantage of such features, as demonstrated by the multiple successes of related methods on the MVTec Anomaly Detection (MVTec AD) dataset. These methods make use of neural networks pre-trained on auxiliary classification tasks such as ImageNet. However, to our knowledge, no comparative study of robustness to the low data regimes between these approaches has been conducted yet. For quality inspection applications, the handling of limited sample sizes may be crucial as large quantities of images are not available for small series. In this work, we aim to compare three approaches based on deep pre-trained features when varying the quantity of available data in MVTec AD: KNN, Mahalanobis, and PaDiM. We show that although these methods are mostly robust to small sample sizes, they still can benefit greatly from using data augmentation in the original image space, which allows to deal with very small production runs.