# AssemAI: Interpretable Image-Based Anomaly Detection for Manufacturing Pipelines

arXiv: https://arxiv.org/abs/2408.02181 · HTML: https://mlsanity.com/paper/2408.02181/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 49 references that are in the map.
- First posted on arXiv 5 Aug 2024.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/1910.01279/) (2019)
- [YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information](https://mlsanity.com/paper/2402.13616/) (2024)
- [Zero-Shot Object Detection](https://mlsanity.com/paper/1804.04340/) (2018)
- [Deep Learning for Anomaly Detection: A Review](https://mlsanity.com/paper/2007.02500/) (2020)
- [IM-IAD: Industrial Image Anomaly Detection Benchmark in Manufacturing](https://mlsanity.com/paper/2301.13359/) (2023)
- [Dont Even Look Once: Synthesizing Features for Zero-Shot Detection](https://mlsanity.com/paper/1911.07933/) (2019)
- [Zero-Shot Object Detection: Learning to Simultaneously Recognize and Localize Novel Concepts](https://mlsanity.com/paper/1803.06049/) (2018)
- [Synthesizing the Unseen for Zero-shot Object Detection](https://mlsanity.com/paper/2010.09425/) (2020)
- [Background Learnable Cascade for Zero-Shot Object Detection](https://mlsanity.com/paper/2010.04502/) (2020)
- [Anomaly Detection using Deep Learning based Image Completion](https://mlsanity.com/paper/1811.06861/) (2018)
- [Analog and Multi-modal Manufacturing Datasets Acquired on the Future Factories Platform](https://mlsanity.com/paper/2401.15544/) (2024)
- [Composite Score for Anomaly Detection in Imbalanced Real-World Industrial Dataset](https://mlsanity.com/paper/2211.15513/) (2022)
- [Anomaly Detection in Images](https://mlsanity.com/paper/1905.13147/) (2019)
- [An Encoder-Decoder Based Approach for Anomaly Detection with Application in Additive Manufacturing](https://mlsanity.com/paper/1907.11778/) (2019)
- [Generalizable Industrial Visual Anomaly Detection with Self-Induction Vision Transformer](https://mlsanity.com/paper/2211.12311/) (2022)

## Built on by (most cited first, 3 of 3)

- [NSF-MAP: Neurosymbolic Multimodal Fusion for Robust and Interpretable Anomaly Prediction in Assembly Pipelines](https://mlsanity.com/paper/2505.06333/) (2025)
- [Hard to See, Hard to Label: Generative and Symbolic Acquisition for Subtle Visual Phenomena](https://mlsanity.com/paper/2604.22990/) (2026)
- [Where Should Knowledge Enter? A Layered Framework for Knowledge Infusion in Multimodal Iterative Generative Model](https://mlsanity.com/paper/2606.06356/) (2026)

## Abstract (arXiv)

Anomaly detection in manufacturing pipelines remains a critical challenge, intensified by the complexity and variability of industrial environments. This paper introduces AssemAI, an interpretable image-based anomaly detection system tailored for smart manufacturing pipelines. Utilizing a curated image dataset from an industry-focused rocket assembly pipeline, we address the challenge of imbalanced image data and demonstrate the importance of image-based methods in anomaly detection. Our primary contributions include deriving an image dataset, fine-tuning an object detection model YOLO-FF, and implementing a custom anomaly detection model for assembly pipelines. The proposed approach leverages domain knowledge in data preparation, model development and reasoning. We implement several anomaly detection models on the derived image dataset, including a Convolutional Neural Network, Vision Transformer (ViT), and pre-trained versions of these models. Additionally, we incorporate explainability techniques at both user and model levels, utilizing ontology for user-level explanations and SCORE-CAM for in-depth feature and model analysis. Finally, the best-performing anomaly detection model and YOLO-FF are deployed in a real-time setting. Our results include ablation studies on the baselines and a comprehensive evaluation of the proposed system. This work highlights the broader impact of advanced image-based anomaly detection in enhancing the reliability and efficiency of smart manufacturing processes. The image dataset, codes to reproduce the results and additional experiments are available at https://github.com/renjithk4/AssemAI.