# Active Learning and Novel Model Calibration Measurements for Automated Visual Inspection in Manufacturing

arXiv: https://arxiv.org/abs/2209.05486 · HTML: https://mlsanity.com/paper/2209.05486/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 76 references that are in the map.
- First posted on arXiv 12 Sep 2022.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [CutPaste: Self-Supervised Learning for Anomaly Detection and Localization](https://mlsanity.com/paper/2104.04015/) (2021)
- [DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detection](https://mlsanity.com/paper/2108.07610/) (2021)
- [Measuring Calibration in Deep Learning](https://mlsanity.com/paper/1904.01685/) (2019)
- [Verified Uncertainty Calibration](https://mlsanity.com/paper/1909.10155/) (2019)
- [Sub-Image Anomaly Detection with Deep Pyramid Correspondences](https://mlsanity.com/paper/2005.02357/) (2020)
- [Multivariate Confidence Calibration for Object Detection](https://mlsanity.com/paper/2004.13546/) (2020)
- [DSR -- A dual subspace re-projection network for surface anomaly detection](https://mlsanity.com/paper/2208.01521/) (2022)
- [A Review of Feature Selection Methods Based on Mutual Information](https://mlsanity.com/paper/1509.07577/) (2015)
- [Distribution-free calibration guarantees for histogram binning without sample splitting](https://mlsanity.com/paper/2105.04656/) (2021)
- [Estimating Expected Calibration Errors](https://mlsanity.com/paper/2109.03480/) (2021)
- [Human-Centric Artificial Intelligence Architecture for Industry 5.0 Applications](https://mlsanity.com/paper/2203.10794/) (2022)
- [Probability Calibration Trees](https://mlsanity.com/paper/1808.00111/) (2018)
- [Active learning using weakly supervised signals for quality inspection](https://mlsanity.com/paper/2104.02973/) (2021)
- [Streaming Machine Learning and Online Active Learning for Automated Visual Inspection](https://mlsanity.com/paper/2110.09396/) (2021)
- [Towards Active Learning Based Smart Assistant for Manufacturing](https://mlsanity.com/paper/2103.16177/) (2021)

## Built on by (most cited first, 4 of 4)

- [Bayesian Quadrature for Neural Ensemble Search](https://mlsanity.com/paper/2303.08874/) (2023)
- [Human in the AI loop via xAI and Active Learning for Visual Inspection](https://mlsanity.com/paper/2307.05508/) (2023)
- [A Data-Efficient Sequential Learning Framework for Melt Pool Defect Classification in Laser Powder Bed Fusion](https://mlsanity.com/paper/2411.10822/) (2024)
- [Calibrated Uncertainty Sampling for Active Learning](https://mlsanity.com/paper/2510.03162/) (2025)

## Abstract (arXiv)

Quality control is a crucial activity performed by manufacturing enterprises to ensure that their products meet quality standards and avoid potential damage to the brand's reputation. The decreased cost of sensors and connectivity enabled increasing digitalization of manufacturing. In addition, artificial intelligence enables higher degrees of automation, reducing overall costs and time required for defect inspection. This research compares three active learning approaches, having single and multiple oracles, to visual inspection. Six new metrics are proposed to assess the quality of calibration without the need for ground truth. Furthermore, this research explores whether existing calibrators can improve their performance by leveraging an approximate ground truth to enlarge the calibration set. The experiments were performed on real-world data provided by Philips Consumer Lifestyle BV. Our results show that the explored active learning settings can reduce the data labeling effort by between three and four percent without detriment to the overall quality goals, considering a threshold of p=0.95. Furthermore, the results show that the proposed calibration metrics successfully capture relevant information otherwise available to metrics used up to date only through ground truth data. Therefore, the proposed metrics can be used to estimate the quality of models' probability calibration without committing to a labeling effort to obtain ground truth data.