# MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection

arXiv: https://arxiv.org/abs/2410.09453 · HTML: https://mlsanity.com/paper/2410.09453/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 46 of its 60 references that are in the map.
- First posted on arXiv 12 Oct 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 30 of 46)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [Are We on the Right Way for Evaluating Large Vision-Language Models?](https://mlsanity.com/paper/2403.20330/) (2024)
- [Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs](https://mlsanity.com/paper/2406.16860/) (2024)
- [LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models](https://mlsanity.com/paper/2407.07895/) (2024)
- [Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset](https://mlsanity.com/paper/2402.14804/) (2024)
- [DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detection](https://mlsanity.com/paper/2108.07610/) (2021)
- [Anomaly Detection via Reverse Distillation from One-Class Embedding](https://mlsanity.com/paper/2201.10703/) (2022)
- [SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation](https://mlsanity.com/paper/2207.14315/) (2022)
- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering](https://mlsanity.com/paper/2305.10415/) (2023)
- [InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model](https://mlsanity.com/paper/2401.16420/) (2024)
- [SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models](https://mlsanity.com/paper/2311.07575/) (2023)
- [AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection](https://mlsanity.com/paper/2310.18961/) (2023)
- [Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future](https://mlsanity.com/paper/2309.15402/) (2023)
- [GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models](https://mlsanity.com/paper/2303.10130/) (2023)
- [Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detection](https://mlsanity.com/paper/2403.12580/) (2024)
- [Efficient Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2405.10739/) (2024)

## Built on by (most cited first, 30 of 47)

- [Can Multimodal Large Language Models be Guided to Improve Industrial Anomaly Detection?](https://mlsanity.com/paper/2501.15795/) (2025)
- [IAD-R1: Reinforcing Consistent Reasoning in Industrial Anomaly Detection](https://mlsanity.com/paper/2508.09178/) (2025)
- [EMIT: Enhancing MLLMs for Industrial Anomaly Detection via Difficulty-Aware GRPO](https://mlsanity.com/paper/2507.21619/) (2025)
- [Normal-Abnormal Guided Generalist Anomaly Detection](https://mlsanity.com/paper/2510.00495/) (2025)
- [LR-IAD:Mask-Free Industrial Anomaly Detection with Logical Reasoning](https://mlsanity.com/paper/2504.19524/) (2025)
- [Triad: Empowering LMM-based Anomaly Detection with Vision Expert-guided Visual Tokenizer and Manufacturing Process](https://mlsanity.com/paper/2503.13184/) (2025)
- [Kaputt: A Large-Scale Dataset for Visual Defect Detection](https://mlsanity.com/paper/2510.05903/) (2025)
- [Towards Explainable Industrial Anomaly Detection via Knowledge-Guided Latent Reasoning](https://mlsanity.com/paper/2602.09850/) (2026)
- [AD-Copilot: A Vision-Language Assistant for Industrial Anomaly Detection via Visual In-context Comparison](https://mlsanity.com/paper/2603.13779/) (2026)
- [AgentIAD: Agentic Industrial Anomaly Detection via Adaptive Memory Augmentation](https://mlsanity.com/paper/2512.13671/) (2025)
- [VLM-SubtleBench: How Far Are VLMs from Human-Level Subtle Comparative Reasoning?](https://mlsanity.com/paper/2603.07888/) (2026)
- [JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QA](https://mlsanity.com/paper/2605.20284/) (2026)
- [Multimodal RAG-driven Anomaly Detection and Classification in Laser Powder Bed Fusion using Large Language Models](https://mlsanity.com/paper/2505.13828/) (2025)
- [The Side Effects of Being Smart: Safety Risks in MLLMs' Multi-Image Reasoning](https://mlsanity.com/paper/2601.14127/) (2026)
- [Real-IAD Variety: Pushing Industrial Anomaly Detection Dataset to a Modern Era](https://mlsanity.com/paper/2511.00540/) (2025)
- [EAGLE: Expert-Augmented Attention Guidance for Tuning-Free Industrial Anomaly Detection in Multimodal Large Language Models](https://mlsanity.com/paper/2602.17419/) (2026)
- [VitaTouch: Property-Aware Vision-Tactile-Language Model for Robotic Quality Inspection in Manufacturing](https://mlsanity.com/paper/2604.03322/) (2026)
- [FORGE: Fine-grained Multimodal Evaluation for Manufacturing Scenarios](https://mlsanity.com/paper/2604.07413/) (2026)
- [ASBench: Image Anomalies Synthesis Benchmark for Anomaly Detection](https://mlsanity.com/paper/2510.07927/) (2025)
- [MicroVQA++: High-Quality Microscopy Reasoning Dataset with Weakly Supervised Graphs for Multimodal Large Language Model](https://mlsanity.com/paper/2511.11407/) (2025)
- [ABounD: Adversarial Boundary-Driven Few-Shot Learning for Multi-Class Anomaly Detection](https://mlsanity.com/paper/2511.22436/) (2025)
- [M3-AD: Reflection-aware Multi-modal, Multi-category, and Multi-dimensional Benchmark and Framework for Industrial Anomaly Detection](https://mlsanity.com/paper/2603.00055/) (2026)
- [MIDAS: Multi-Image Dispersion and Semantic Reconstruction for Jailbreaking MLLMs](https://mlsanity.com/paper/2603.00565/) (2026)
- [AG-VAS: Anchor-Guided Zero-Shot Visual Anomaly Segmentation with Large Multimodal Models](https://mlsanity.com/paper/2603.01305/) (2026)
- [Topo-R1: Detecting Topological Anomalies via Vision-Language Models](https://mlsanity.com/paper/2603.13054/) (2026)
- [AnomalyAgent: Training-Free Agentic Models for Zero-/Few-Shot Anomaly Detection](https://mlsanity.com/paper/2605.30140/) (2026)
- [OPD-IAD: From Language Judgment to Industrial Anomaly Detection via On-Policy Self-Distillation](https://mlsanity.com/paper/2607.18850/) (2026)
- [VD-DeepStack: Bridging Visual Comparison and Language Reasoning for Few-Shot Anomaly Detection](https://mlsanity.com/paper/2609.34949/) (2026)
- [ADSeeker: A Knowledge-Grounded Reasoning Framework for Industry Anomaly Detection and Reasoning](https://mlsanity.com/paper/2508.03088/) (2025)
- [PiERN: Token-Level Routing for Integrating High-Precision Computation and Reasoning](https://mlsanity.com/paper/2509.18169/) (2025)

## Abstract (arXiv)

In the field of industrial inspection, Multimodal Large Language Models (MLLMs) have a high potential to renew the paradigms in practical applications due to their robust language capabilities and generalization abilities. However, despite their impressive problem-solving skills in many domains, MLLMs' ability in industrial anomaly detection has not been systematically studied. To bridge this gap, we present MMAD, the first-ever full-spectrum MLLMs benchmark in industrial Anomaly Detection. We defined seven key subtasks of MLLMs in industrial inspection and designed a novel pipeline to generate the MMAD dataset with 39,672 questions for 8,366 industrial images. With MMAD, we have conducted a comprehensive, quantitative evaluation of various state-of-the-art MLLMs. The commercial models performed the best, with the average accuracy of GPT-4o models reaching 74.9%. However, this result falls far short of industrial requirements. Our analysis reveals that current MLLMs still have significant room for improvement in answering questions related to industrial anomalies and defects. We further explore two training-free performance enhancement strategies to help models improve in industrial scenarios, highlighting their promising potential for future research.