# A Review of Benchmarks for Visual Defect Detection in the Manufacturing Industry

arXiv: https://arxiv.org/abs/2305.13261 · HTML: https://mlsanity.com/paper/2305.13261/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 21 references that are in the map.
- First posted on arXiv 5 May 2023.

## Builds on (most cited first, 7 of 7)

- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detection](https://mlsanity.com/paper/2108.07610/) (2021)
- [Segmentation-Based Deep-Learning Approach for Surface-Defect Detection](https://mlsanity.com/paper/1903.08536/) (2019)
- [Mixed supervision for surface-defect detection: from weakly to fully supervised learning](https://mlsanity.com/paper/2104.06064/) (2021)
- [Optimization for Medical Image Segmentation: Theory and Practice when evaluating with Dice Score or Jaccard Index](https://mlsanity.com/paper/2010.13499/) (2020)
- [End-to-end training of a two-stage neural network for defect detection](https://mlsanity.com/paper/2007.07676/) (2020)
- [Unsupervised machine learning via transfer learning and k-means clustering to classify materials image data](https://mlsanity.com/paper/2007.08361/) (2020)

## Built on by (most cited first, 1 of 1)

- [Defect Spectrum: A Granular Look of Large-Scale Defect Datasets with Rich Semantics](https://mlsanity.com/paper/2310.17316/) (2023)

## Abstract (arXiv)

The field of industrial defect detection using machine learning and deep learning is a subject of active research. Datasets, also called benchmarks, are used to compare and assess research results. There is a number of datasets in industrial visual inspection, of varying quality. Thus, it is a difficult task to determine which dataset to use. Generally speaking, datasets which include a testing set, with precise labeling and made in real-world conditions should be preferred. We propose a study of existing benchmarks to compare and expose their characteristics and their use-cases. A study of industrial metrics requirements, as well as testing procedures, will be presented and applied to the studied benchmarks. We discuss our findings by examining the current state of benchmarks for industrial visual inspection, and by exposing guidelines on the usage of benchmarks.