# Can Multimodal Large Language Models be Guided to Improve Industrial Anomaly Detection?

arXiv: https://arxiv.org/abs/2501.15795 · HTML: https://mlsanity.com/paper/2501.15795/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 16 of its 41 references that are in the map.
- First posted on arXiv 27 Jan 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 16 of 16)

- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://mlsanity.com/paper/2404.16821/) (2024)
- [Multimodal Chain-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2302.00923/) (2023)
- [AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection](https://mlsanity.com/paper/2310.18961/) (2023)
- [Simple synthetic data reduces sycophancy in large language models](https://mlsanity.com/paper/2308.03958/) (2023)
- [RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model](https://mlsanity.com/paper/2402.10828/) (2024)
- [Retrieving Multimodal Information for Augmented Generation: A Survey](https://mlsanity.com/paper/2303.10868/) (2023)
- [A Survey on Visual Anomaly Detection: Challenge, Approach, and Prospect](https://mlsanity.com/paper/2401.16402/) (2024)
- [MLLM Is a Strong Reranker: Advancing Multimodal Retrieval-augmented Generation via Knowledge-enhanced Reranking and Noise-injected Training](https://mlsanity.com/paper/2407.21439/) (2024)
- [MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection](https://mlsanity.com/paper/2410.09453/) (2024)
- [Myriad: Large Multimodal Model by Applying Vision Experts for Industrial Anomaly Detection](https://mlsanity.com/paper/2310.19070/) (2023)
- [RAR: Retrieving And Ranking Augmented MLLMs for Visual Recognition](https://mlsanity.com/paper/2403.13805/) (2024)
- [VMAD: Visual-enhanced Multimodal Large Language Model for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2409.20146/) (2024)

## Built on by (most cited first, 17 of 17)

- [AnomalyR1: A GRPO-based End-to-end MLLM for Industrial Anomaly Detection](https://mlsanity.com/paper/2504.11914/) (2025)
- [OmniAD: Detect and Understand Industrial Anomaly via Multimodal Reasoning](https://mlsanity.com/paper/2505.22039/) (2025)
- [IAD-R1: Reinforcing Consistent Reasoning in Industrial Anomaly Detection](https://mlsanity.com/paper/2508.09178/) (2025)
- [EMIT: Enhancing MLLMs for Industrial Anomaly Detection via Difficulty-Aware GRPO](https://mlsanity.com/paper/2507.21619/) (2025)
- [LR-IAD:Mask-Free Industrial Anomaly Detection with Logical Reasoning](https://mlsanity.com/paper/2504.19524/) (2025)
- [Detect, Classify, Act: Categorizing Industrial Anomalies with Multi-Modal Large Language Models](https://mlsanity.com/paper/2505.02626/) (2025)
- [A Survey on Foundation-Model-Based Industrial Defect Detection](https://mlsanity.com/paper/2502.19106/) (2025)
- [AD-Copilot: A Vision-Language Assistant for Industrial Anomaly Detection via Visual In-context Comparison](https://mlsanity.com/paper/2603.13779/) (2026)
- [Latent Anomaly Knowledge Excavation: Unveiling Sparse Sensitive Neurons in Vision-Language Models](https://mlsanity.com/paper/2604.07802/) (2026)
- [EAGLE: Expert-Augmented Attention Guidance for Tuning-Free Industrial Anomaly Detection in Multimodal Large Language Models](https://mlsanity.com/paper/2602.17419/) (2026)
- [ABounD: Adversarial Boundary-Driven Few-Shot Learning for Multi-Class Anomaly Detection](https://mlsanity.com/paper/2511.22436/) (2025)
- [Task-Aware Scanning Parameter Configuration for Robotic Inspection Using Vision Language Embeddings and Hyperdimensional Computing](https://mlsanity.com/paper/2605.03909/) (2026)
- [PB-IAD: Utilizing multimodal foundation models for semantic industrial anomaly detection in dynamic manufacturing environments](https://mlsanity.com/paper/2508.14504/) (2025)
- [Explainable Deep Convolutional Multi-Type Anomaly Detection](https://mlsanity.com/paper/2511.11165/) (2025)
- [Physics-Grounded Multi-Agent Architecture for Traceable, Risk-Aware Human-AI Decision Support in Manufacturing](https://mlsanity.com/paper/2605.04003/) (2026)
- [DifferAD-R1: A Difference-Guided IndustrialAnomaly Localization with Multimodal LargeLanguage Models](https://mlsanity.com/paper/2606.16601/) (2026)
- [Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detection](https://mlsanity.com/paper/2607.03817/) (2026)

## Abstract (arXiv)

In industrial settings, the accurate detection of anomalies is essential for maintaining product quality and ensuring operational safety. Traditional industrial anomaly detection (IAD) models often struggle with flexibility and adaptability, especially in dynamic production environments where new defect types and operational changes frequently arise. Recent advancements in Multimodal Large Language Models (MLLMs) hold promise for overcoming these limitations by combining visual and textual information processing capabilities. MLLMs excel in general visual understanding due to their training on large, diverse datasets, but they lack domain-specific knowledge, such as industry-specific defect tolerance levels, which limits their effectiveness in IAD tasks. To address these challenges, we propose Echo, a novel multi-expert framework designed to enhance MLLM performance for IAD. Echo integrates four expert modules: Reference Extractor which provides a contextual baseline by retrieving similar normal images, Knowledge Guide which supplies domain-specific insights, Reasoning Expert which enables structured, stepwise reasoning for complex queries, and Decision Maker which synthesizes information from all modules to deliver precise, context-aware responses. Evaluated on the MMAD benchmark, Echo demonstrates significant improvements in adaptability, precision, and robustness, moving closer to meeting the demands of real-world industrial anomaly detection.