mlsanityData to 14 Oct 2026

MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection

arXiv 2410.09453 ↗First posted 12 Oct 2024cs.AI cs.CVcited by 47

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

In the field of industrial inspection, Multimodal Large Language Models (MLLMs) have a high potential to renew the paradigms in practical applications due to their robust language capabilities and generalization abilities. However, despite their impressive problem-solving skills in many domains, MLLMs' ability in industrial anomaly detection has not been systematically studied. To bridge this gap, we present MMAD, the first-ever full-spectrum MLLMs benchmark in industrial Anomaly Detection. We defined seven key subtasks of MLLMs in industrial inspection and designed a novel pipeline to generate the MMAD dataset with 39,672 questions for 8,366 industrial images. With MMAD, we have conducted a comprehensive, quantitative evaluation of various state-of-the-art MLLMs. The commercial models performed the best, with the average accuracy of GPT-4o models reaching 74.9%. However, this result falls far short of industrial requirements. Our analysis reveals that current MLLMs still have significant room for improvement in answering questions related to industrial anomalies and defects. We further explore two training-free performance enhancement strategies to help models improve in industrial scenarios, highlighting their promising potential for future research.

Key facts. Cited by 47 papers in the map (data through 14 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 46 of its 60 references that are in the map. First posted on arXiv 12 Oct 2024.

Builds on

46 of 60 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023GPT-4V-AD: Exploring Grounding Potential of VQA-oriented GPT-4V for Zero-shot Anomaly Detection5 Nov
  2. 2023Myriad: Large Multimodal Model by Applying Vision Experts for Industrial Anomaly Detection29 Oct

Other references · 44

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  6. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  7. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  8. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  9. 2019Towards VQA Models That Can Readcited by 1,718
  10. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  11. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  12. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
32 more
  1. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  2. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  3. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  4. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  5. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
  6. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
  7. 2021DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detectioncited by 514
  8. 2022Anomaly Detection via Reverse Distillation from One-Class Embeddingcited by 421
  9. 2022SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentationcited by 400
  10. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 354
  11. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
  12. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  13. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 286
  14. 2023AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detectioncited by 271
  15. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  16. 2023GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Modelscited by 176
  17. 2024Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detectioncited by 139
  18. 2024Efficient Multimodal Large Language Models: A Surveycited by 96
  19. 2024GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AIcited by 96
  20. 2024Toward Generalist Anomaly Detection via In-context Residual Learning with Few-shot Sample Promptscited by 89
  21. 2024A Survey on Benchmarks of Multimodal Large Language Modelscited by 82
  22. 2024SoftPatch: Unsupervised Anomaly Detection with Noisy Datacited by 55
  23. 2023Towards Generic Anomaly Detection and Understanding: Large-scale Visual-linguistic Model (GPT-4V) Takes the Leadcited by 40
  24. 2024M3DM-NR: RGB-3D Noisy-Resistant Industrial Anomaly Detection via Multimodal Denoisingcited by 24
  25. 2023PKU-GoodsAD: A Supermarket Goods Dataset for Unsupervised Anomaly Detection and Segmentationcited by 21
  26. 2024Customizing Visual-Language Foundation Models for Multi-modal Anomaly Detection and Reasoningcited by 20
  27. 2024LogiCode: an LLM-Driven Framework for Logical Anomaly Detectioncited by 19
  28. 2023Defect Spectrum: A Granular Look of Large-Scale Defect Datasets with Rich Semanticscited by 16
  29. 2024A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Modelscited by 16
  30. 2024FabGPT: An Efficient Large Multimodal Model for Complex Wafer Defect Knowledge Queriescited by 12
  31. 2022A Survey of Visual Sensory Anomaly Detectioncited by 11
  32. 2024Toward Multi-class Anomaly Detection: Exploring Class-aware Unified Model against Inter-class Interferencecited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 16; 2026: 31.

47 papers.

Sort:

By area:

  1. 2025Can Multimodal Large Language Models be Guided to Improve Industrial Anomaly Detection?cited by 17
  2. 2025IAD-R1: Reinforcing Consistent Reasoning in Industrial Anomaly Detectioncited by 17
  3. 2025EMIT: Enhancing MLLMs for Industrial Anomaly Detection via Difficulty-Aware GRPOcited by 12
  4. 2025Normal-Abnormal Guided Generalist Anomaly Detectioncited by 10
  5. 2025LR-IAD:Mask-Free Industrial Anomaly Detection with Logical Reasoningcited by 9
  6. 2025Triad: Empowering LMM-based Anomaly Detection with Vision Expert-guided Visual Tokenizer and Manufacturing Processcited by 8
  7. 2025Kaputt: A Large-Scale Dataset for Visual Defect Detectioncited by 6
  8. 2026Towards Explainable Industrial Anomaly Detection via Knowledge-Guided Latent Reasoningcited by 6
  9. 2026AD-Copilot: A Vision-Language Assistant for Industrial Anomaly Detection via Visual In-context Comparisoncited by 5
  10. 2025AgentIAD: Agentic Industrial Anomaly Detection via Adaptive Memory Augmentationcited by 4
  11. 2026VLM-SubtleBench: How Far Are VLMs from Human-Level Subtle Comparative Reasoning?cited by 4
  12. 2026JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QAcited by 4
35 more
  1. 2025Multimodal RAG-driven Anomaly Detection and Classification in Laser Powder Bed Fusion using Large Language Modelscited by 3
  2. 2026The Side Effects of Being Smart: Safety Risks in MLLMs' Multi-Image Reasoningcited by 3
  3. 2025Real-IAD Variety: Pushing Industrial Anomaly Detection Dataset to a Modern Eracited by 2
  4. 2026EAGLE: Expert-Augmented Attention Guidance for Tuning-Free Industrial Anomaly Detection in Multimodal Large Language Modelscited by 2
  5. 2026VitaTouch: Property-Aware Vision-Tactile-Language Model for Robotic Quality Inspection in Manufacturingcited by 2
  6. 2026FORGE: Fine-grained Multimodal Evaluation for Manufacturing Scenarioscited by 2
  7. 2025ASBench: Image Anomalies Synthesis Benchmark for Anomaly Detectioncited by 1
  8. 2025MicroVQA++: High-Quality Microscopy Reasoning Dataset with Weakly Supervised Graphs for Multimodal Large Language Modelcited by 1
  9. 2025ABounD: Adversarial Boundary-Driven Few-Shot Learning for Multi-Class Anomaly Detectioncited by 1
  10. 2026M3-AD: Reflection-aware Multi-modal, Multi-category, and Multi-dimensional Benchmark and Framework for Industrial Anomaly Detectioncited by 1
  11. 2026MIDAS: Multi-Image Dispersion and Semantic Reconstruction for Jailbreaking MLLMscited by 1
  12. 2026AG-VAS: Anchor-Guided Zero-Shot Visual Anomaly Segmentation with Large Multimodal Modelscited by 1
  13. 2026Topo-R1: Detecting Topological Anomalies via Vision-Language Modelscited by 1
  14. 2026AnomalyAgent: Training-Free Agentic Models for Zero-/Few-Shot Anomaly Detectioncited by 1
  15. 2026OPD-IAD: From Language Judgment to Industrial Anomaly Detection via On-Policy Self-Distillationcited by 1
  16. 2026VD-DeepStack: Bridging Visual Comparison and Language Reasoning for Few-Shot Anomaly Detectioncited by 1
  17. 2025ADSeeker: A Knowledge-Grounded Reasoning Framework for Industry Anomaly Detection and Reasoningcited by 0
  18. 2025PiERN: Token-Level Routing for Integrating High-Precision Computation and Reasoningcited by 0
  19. 2025TokenCLIP: Token-wise Prompt Learning for Zero-shot Anomaly Detectioncited by 0
  20. 2026SteelDefectX: A Multi-Form Vision-Language Dataset and Benchmark for Steel Surface Defect Analysiscited by 0
  21. 2026MARINER: A 3E-Driven Benchmark for Fine-Grained Perception and Complex Reasoning in Open-Water Environmentscited by 0
  22. 2026Large-Scale Universal Defect Generation: Foundation Models and Datasetscited by 0
  23. 2026Out of Context: Reliability in Multimodal Anomaly Detection Requires Contextual Inferencecited by 0
  24. 2026AnomalyClaw: A Universal Visual Anomaly Detection Agent via Tool-Grounded Refutationcited by 0
  25. 2026MMVIAD: Multi-view Multi-task Video Understanding for Industrial Anomaly Detectioncited by 0
  26. 2026A Structured Benchmark for Text-Guided Anomaly Detection: When Language Stops Conditioning the Decisioncited by 0
  27. 2026DifferAD-R1: A Difference-Guided IndustrialAnomaly Localization with Multimodal LargeLanguage Modelscited by 0
  28. 2026Vision-driven Preference Synthesis for Mitigating Hallucinations in VLMscited by 0
  29. 2026Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detectioncited by 0
  30. 2026RobustMAD: Evaluating Real-World Robustness of Multimodal Small Language Models for Deployable Anomaly Detection Assistantscited by 0
  31. 2026O-VAD: Industrial Video Anomaly Detection through Object-Centric Tracking and Reasoningcited by 0
  32. 2026ADOPD: Reference-Privileged On-Policy Distillation for MLLM-Based Industrial Anomaly Detectioncited by 0
  33. 2026GSO-Net: Visual State Machines for Hazardous Freight Transfer Compliance at Petrochemical Logistics Nodescited by 0
  34. 2026Industrial Anomaly Detection via Defect-Grounded Reasoning in Visual Latent Spacecited by 0
  35. 2026TED:Text-Axis Evidence Decomposition for Prompted Anomaly Localizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025IAD-GPT: Advancing Visual Knowledge in Multimodal Large Language Model for Industrial Anomaly Detection
  2. 2026Unification of Closed-Open Industrial Detection Scenarios: New Large-Scale Benchmarks,Challenges and Baselines
  3. 2024VMAD: Visual-enhanced Multimodal Large Language Model for Zero-Shot Anomaly Detection
  4. 2026MMR-AD: A Large-Scale Multimodal Dataset for Benchmarking General Anomaly Detection with Multimodal Large Language Models
  5. 2025Detect, Classify, Act: Categorizing Industrial Anomalies with Multi-Modal Large Language Models
  6. 2026Is In-Domain Training Enough for Fine-Grained Industrial Anomaly Understanding?
  7. 2023AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models
  8. 2025AnomalyR1: A GRPO-based End-to-end MLLM for Industrial Anomaly Detection