# From Benchmark Performance to Tool Deployment: Human-in-the-Loop Anomaly Detection

arXiv: https://arxiv.org/abs/2608.07770 · HTML: https://mlsanity.com/paper/2608.07770/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 34 references that are in the map.
- First posted on arXiv 7 Aug 2026.

## Builds on (most cited first, 24 of 24)

- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization](https://mlsanity.com/paper/2011.08785/) (2020)
- [DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detection](https://mlsanity.com/paper/2108.07610/) (2021)
- [Anomaly Detection via Reverse Distillation from One-Class Embedding](https://mlsanity.com/paper/2201.10703/) (2022)
- [GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training](https://mlsanity.com/paper/1805.06725/) (2018)
- [CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows](https://mlsanity.com/paper/2107.12571/) (2021)
- [FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows](https://mlsanity.com/paper/2111.07677/) (2021)
- [The VIA Annotation Software for Images, Audio and Video](https://mlsanity.com/paper/1904.10699/) (2019)
- [EfficientAD: Accurate Visual Anomaly Detection at Millisecond-Level Latencies](https://mlsanity.com/paper/2303.14535/) (2023)
- [Deep Industrial Image Anomaly Detection: A Survey](https://mlsanity.com/paper/2301.11514/) (2023)
- [Student-Teacher Feature Pyramid Matching for Anomaly Detection](https://mlsanity.com/paper/2103.04257/) (2021)
- [CFA: Coupled-hypersphere-based Feature Adaptation for Target-Oriented Anomaly Localization](https://mlsanity.com/paper/2206.04325/) (2022)
- [Fully Convolutional Cross-Scale-Flows for Image-based Defect Detection](https://mlsanity.com/paper/2110.02855/) (2021)
- [Dinomaly: The Less Is More Philosophy in Multi-Class Unsupervised Anomaly Detection](https://mlsanity.com/paper/2405.14325/) (2024)
- [DSR -- A dual subspace re-projection network for surface anomaly detection](https://mlsanity.com/paper/2208.01521/) (2022)
- [Anomalib: A Deep Learning Library for Anomaly Detection](https://mlsanity.com/paper/2202.08341/) (2022)
- [IM-IAD: Industrial Image Anomaly Detection Benchmark in Manufacturing](https://mlsanity.com/paper/2301.13359/) (2023)
- [AnomalyDINO: Boosting Patch-based Few-shot Anomaly Detection with DINOv2](https://mlsanity.com/paper/2405.14529/) (2024)
- [Probabilistic Modeling of Deep Features for Out-of-Distribution and Adversarial Detection](https://mlsanity.com/paper/1909.11786/) (2019)
- [U-Flow: A U-shaped Normalizing Flow for Anomaly Detection with Unsupervised Threshold](https://mlsanity.com/paper/2211.12353/) (2022)
- [SuperSimpleNet: Unifying Unsupervised and Supervised Learning for Fast and Reliable Surface Defect Detection](https://mlsanity.com/paper/2408.03143/) (2024)
- [What makes a good data augmentation for few-shot unsupervised image anomaly detection?](https://mlsanity.com/paper/2304.03294/) (2023)
- [FRE: A Fast Method For Anomaly Detection And Segmentation](https://mlsanity.com/paper/2211.12650/) (2022)

## Abstract (arXiv)

Automated anomaly detection methods often report strong performance on curated academic benchmarks, but their behavior under real-world industrial conditions is less clear. In this work, we evaluate 19 unsupervised anomaly detection models on the BowTie dataset, a challenging manufacturing dataset with reflective surfaces, subtle defects, and profile-specific variation. In contrast to benchmark results, we observe that model performance is less stable than typically reported on standard benchmarks such as MVTec AD, highly sensitive to preprocessing, and inconsistent across conditions, with no single approach emerging as uniformly robust; a consensus audit further indicates that nominal-data quality affects deployment. Motivated by these findings, we developed and initially deployed a unified human-in-the-loop framework for manufactured-part inspection that combines image annotation, AI-assisted defect detection, and an integrated validation engine, replacing a prior manual visual inspection and documentation workflow. The system supports heatmap-guided defect review, SAM-refined candidate regions for inspector acceptance, rejection, or boundary adjustment, mask evaluation where annotations exist, and review history for inspector consistency and onboarding. Together, the results highlight the gap between benchmark performance and deployment reality, and provide a practical framework for addressing it.