# Multimodal RAG-driven Anomaly Detection and Classification in Laser Powder Bed Fusion using Large Language Models

arXiv: https://arxiv.org/abs/2505.13828 · HTML: https://mlsanity.com/paper/2505.13828/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 50 references that are in the map.
- First posted on arXiv 20 May 2025.

## Builds on (most cited first, 27 of 27)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation](https://mlsanity.com/paper/2303.14814/) (2023)
- [AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection](https://mlsanity.com/paper/2310.18961/) (2023)
- [VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents](https://mlsanity.com/paper/2410.10594/) (2024)
- [ColPali: Efficient Document Retrieval with Vision Language Models](https://mlsanity.com/paper/2407.01449/) (2024)
- [AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models](https://mlsanity.com/paper/2308.15366/) (2023)
- [Towards Knowledge-Based Recommender Dialog System](https://mlsanity.com/paper/1908.05391/) (2019)
- [Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Models](https://mlsanity.com/paper/2502.07601/) (2025)
- [ClipSAM: CLIP and SAM Collaboration for Zero-Shot Anomaly Segmentation](https://mlsanity.com/paper/2401.12665/) (2024)
- [MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection](https://mlsanity.com/paper/2410.09453/) (2024)
- [Towards LLM-guided Causal Explainability for Black-box Text Classifiers](https://mlsanity.com/paper/2309.13340/) (2023)
- [Myriad: Large Multimodal Model by Applying Vision Experts for Industrial Anomaly Detection](https://mlsanity.com/paper/2310.19070/) (2023)
- [DesignQA: A Multimodal Benchmark for Evaluating Large Language Models' Understanding of Engineering Documentation](https://mlsanity.com/paper/2404.07917/) (2024)
- [How Much Can RAG Help the Reasoning of LLM?](https://mlsanity.com/paper/2410.02338/) (2024)
- [AD-LLM: Benchmarking Large Language Models for Anomaly Detection](https://mlsanity.com/paper/2412.11142/) (2024)
- [M3DM-NR: RGB-3D Noisy-Resistant Industrial Anomaly Detection via Multimodal Denoising](https://mlsanity.com/paper/2406.02263/) (2024)
- [AMGPT: a Large Language Model for Contextual Querying in Additive Manufacturing](https://mlsanity.com/paper/2406.00031/) (2024)
- [LLM-3D Print: Large Language Models To Monitor and Control 3D Printing](https://mlsanity.com/paper/2408.14307/) (2024)
- [VMAD: Visual-enhanced Multimodal Large Language Model for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2409.20146/) (2024)
- [FabGPT: An Efficient Large Multimodal Model for Complex Wafer Defect Knowledge Queries](https://mlsanity.com/paper/2407.10810/) (2024)
- [KAnoCLIP: Zero-Shot Anomaly Detection through Knowledge-Driven Prompt Learning and Enhanced Cross-Modal Integration](https://mlsanity.com/paper/2501.03786/) (2025)
- [AdditiveLLM: Large Language Models Predict Defects in Additive Manufacturing](https://mlsanity.com/paper/2501.17784/) (2025)
- [FDM-Bench: A Comprehensive Benchmark for Evaluating Large Language Models in Additive Manufacturing Tasks](https://mlsanity.com/paper/2412.09819/) (2024)
- [Large Language Models for Extrapolative Modeling of Manufacturing Processes](https://mlsanity.com/paper/2502.12185/) (2025)
- [RAAD-LLM: Adaptive Anomaly Detection Using LLMs and RAG Integration](https://mlsanity.com/paper/2503.02800/) (2025)
- [Keyword-driven Retrieval-Augmented Large Language Models for Cold-start User Recommendations](https://mlsanity.com/paper/2405.19612/) (2024)

## Built on by (most cited first, 3 of 3)

- [MCERF: Advancing Multimodal LLM Evaluation of Engineering Documentation with Enhanced Retrieval](https://mlsanity.com/paper/2604.09552/) (2026)
- [Task-Aware Scanning Parameter Configuration for Robotic Inspection Using Vision Language Embeddings and Hyperdimensional Computing](https://mlsanity.com/paper/2605.03909/) (2026)
- [LLM-ADAM: A Generalizable LLM Agent Framework for Pre-Print Anomaly Detection in Additive Manufacturing](https://mlsanity.com/paper/2605.03328/) (2026)

## Abstract (arXiv)

Additive manufacturing enables the fabrication of complex designs while minimizing waste, but faces challenges related to defects and process anomalies. This study presents a novel multimodal Retrieval-Augmented Generation-based framework that automates anomaly detection across various Additive Manufacturing processes leveraging retrieved information from literature, including images and descriptive text, rather than training datasets. This framework integrates text and image retrieval from scientific literature and multimodal generation models to perform zero-shot anomaly identification, classification, and explanation generation in a Laser Powder Bed Fusion setting. The proposed framework is evaluated on four L-PBF manufacturing datasets from Oak Ridge National Laboratory, featuring various printer makes, models, and materials. This evaluation demonstrates the framework's adaptability and generalizability across diverse images without requiring additional training. Comparative analysis using Qwen2-VL-2B and GPT-4o-mini as MLLM within the proposed framework highlights that GPT-4o-mini outperforms Qwen2-VL-2B and proportional random baseline in manufacturing anomalies classification. Additionally, the evaluation of the RAG system confirms that incorporating retrieval mechanisms improves average accuracy by 12% by reducing the risk of hallucination and providing additional information. The proposed framework can be continuously updated by integrating emerging research, allowing seamless adaptation to the evolving landscape of AM technologies. This scalable, automated, and zero-shot-capable framework streamlines AM anomaly analysis, enhancing efficiency and accuracy.