# Myriad: Large Multimodal Model by Applying Vision Experts for Industrial Anomaly Detection

arXiv: https://arxiv.org/abs/2310.19070 · HTML: https://mlsanity.com/paper/2310.19070/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 31 of its 38 references that are in the map.
- First posted on arXiv 29 Oct 2023.

## Placed in

- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic](https://mlsanity.com/paper/2306.15195/) (2023)
- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [EVA: Exploring the Limits of Masked Visual Representation Learning at Scale](https://mlsanity.com/paper/2211.07636/) (2022)
- [RegionCLIP: Region-based Language-Image Pretraining](https://mlsanity.com/paper/2112.09106/) (2021)
- [PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization](https://mlsanity.com/paper/2011.08785/) (2020)
- [Otter: A Multi-Modal Model with In-Context Instruction Tuning](https://mlsanity.com/paper/2305.03726/) (2023)
- [VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks](https://mlsanity.com/paper/2305.11175/) (2023)
- [Anomaly Detection via Reverse Distillation from One-Class Embedding](https://mlsanity.com/paper/2201.10703/) (2022)
- [SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation](https://mlsanity.com/paper/2207.14315/) (2022)
- [WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation](https://mlsanity.com/paper/2303.14814/) (2023)
- [Sub-Image Anomaly Detection with Deep Pyramid Correspondences](https://mlsanity.com/paper/2005.02357/) (2020)
- [A Unified Model for Multi-class Anomaly Detection](https://mlsanity.com/paper/2206.03687/) (2022)
- [SimpleNet: A Simple Network for Image Anomaly Detection and Localization](https://mlsanity.com/paper/2303.15140/) (2023)
- [AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models](https://mlsanity.com/paper/2308.15366/) (2023)
- [DeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detection](https://mlsanity.com/paper/2211.11317/) (2022)
- [APRIL-GAN: A Zero-/Few-Shot Anomaly Classification and Segmentation Method for CVPR 2023 VAND Workshop Challenge Tracks 1&2: 1st Place on Zero-shot AD and 4th Place on Few-shot AD](https://mlsanity.com/paper/2305.17382/) (2023)
- [Natural Synthetic Anomalies for Self-Supervised Anomaly Detection and Localization](https://mlsanity.com/paper/2109.15222/) (2021)
- [The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open World](https://mlsanity.com/paper/2308.01907/) (2023)
- [Focus the Discrepancy: Intra- and Inter-Correlation Learning for Image Anomaly Detection](https://mlsanity.com/paper/2308.02983/) (2023)
- [RegionBLIP: A Unified Multi-modal Pre-training Framework for Holistic and Regional Comprehension](https://mlsanity.com/paper/2308.02299/) (2023)
- [LafitE: Latent Diffusion Model with Feature Editing for Unsupervised Multi-class Anomaly Detection](https://mlsanity.com/paper/2307.08059/) (2023)

## Built on by (most cited first, 30 of 39)

- [MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection](https://mlsanity.com/paper/2410.09453/) (2024)
- [Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey](https://mlsanity.com/paper/2407.21794/) (2024)
- [AnomalyR1: A GRPO-based End-to-end MLLM for Industrial Anomaly Detection](https://mlsanity.com/paper/2504.11914/) (2025)
- [Large Language Models for Anomaly and Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2409.01980/) (2024)
- [OmniAD: Detect and Understand Industrial Anomaly via Multimodal Reasoning](https://mlsanity.com/paper/2505.22039/) (2025)
- [Can Multimodal Large Language Models be Guided to Improve Industrial Anomaly Detection?](https://mlsanity.com/paper/2501.15795/) (2025)
- [IAD-R1: Reinforcing Consistent Reasoning in Industrial Anomaly Detection](https://mlsanity.com/paper/2508.09178/) (2025)
- [CLIP3D-AD: Extending CLIP for 3D Few-Shot Anomaly Detection with Multi-View Images Generation](https://mlsanity.com/paper/2406.18941/) (2024)
- [A Survey on Safe Multi-Modal Learning System](https://mlsanity.com/paper/2402.05355/) (2024)
- [VMAD: Visual-enhanced Multimodal Large Language Model for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2409.20146/) (2024)
- [EMIT: Enhancing MLLMs for Industrial Anomaly Detection via Difficulty-Aware GRPO](https://mlsanity.com/paper/2507.21619/) (2025)
- [EIAD: Explainable Industrial Anomaly Detection Via Multi-Modal Large Language Models](https://mlsanity.com/paper/2503.14162/) (2025)
- [LAD-Reasoner: Tiny Multimodal Models are Good Reasoners for Logical Anomaly Detection](https://mlsanity.com/paper/2504.12749/) (2025)
- [LR-IAD:Mask-Free Industrial Anomaly Detection with Logical Reasoning](https://mlsanity.com/paper/2504.19524/) (2025)
- [Triad: Empowering LMM-based Anomaly Detection with Vision Expert-guided Visual Tokenizer and Manufacturing Process](https://mlsanity.com/paper/2503.13184/) (2025)
- [Anomaly Detection and Generation with Diffusion Models: A Survey](https://mlsanity.com/paper/2506.09368/) (2025)
- [Foundation Models and Transformers for Anomaly Detection: A Survey](https://mlsanity.com/paper/2507.15905/) (2025)
- [Exploring the Potential of Large Language Models for Improving Digital Forensic Investigation Efficiency](https://mlsanity.com/paper/2402.19366/) (2024)
- [Large Scale Foundation Models for Intelligent Manufacturing Applications: A Survey](https://mlsanity.com/paper/2312.06718/) (2023)
- [Detect, Classify, Act: Categorizing Industrial Anomalies with Multi-Modal Large Language Models](https://mlsanity.com/paper/2505.02626/) (2025)
- [Small Object Few-shot Segmentation for Vision-based Industrial Inspection](https://mlsanity.com/paper/2407.21351/) (2024)
- [A Survey on Foundation-Model-Based Industrial Defect Detection](https://mlsanity.com/paper/2502.19106/) (2025)
- [A VLM-based Method for Visual Anomaly Detection in Robotic Scientific Laboratories](https://mlsanity.com/paper/2506.05405/) (2025)
- [Out-of-Distribution Detection using Synthetic Data Generation](https://mlsanity.com/paper/2502.03323/) (2025)
- [AgentIAD: Agentic Industrial Anomaly Detection via Adaptive Memory Augmentation](https://mlsanity.com/paper/2512.13671/) (2025)
- [Foundation Models for Anomaly Detection: Vision and Challenges](https://mlsanity.com/paper/2502.06911/) (2025)
- [Multimodal RAG-driven Anomaly Detection and Classification in Laser Powder Bed Fusion using Large Language Models](https://mlsanity.com/paper/2505.13828/) (2025)
- [Reasoning-Driven Anomaly Detection and Localization with Image-Level Supervision](https://mlsanity.com/paper/2603.27179/) (2026)
- [Q-Former Autoencoder: A Modern Framework for Medical Anomaly Detection](https://mlsanity.com/paper/2507.18481/) (2025)
- [EAGLE: Expert-Augmented Attention Guidance for Tuning-Free Industrial Anomaly Detection in Multimodal Large Language Models](https://mlsanity.com/paper/2602.17419/) (2026)

## Abstract (arXiv)

Due to the training configuration, traditional industrial anomaly detection (IAD) methods have to train a specific model for each deployment scenario, which is insufficient to meet the requirements of modern design and manufacturing. On the contrary, large multimodal models~(LMMs) have shown eminent generalization ability on various vision tasks, and their perception and comprehension capabilities imply the potential of applying LMMs on IAD tasks. However, we observe that even though the LMMs have abundant knowledge about industrial anomaly detection in the textual domain, the LMMs are unable to leverage the knowledge due to the modality gap between textual and visual domains. To stimulate the relevant knowledge in LMMs and adapt the LMMs towards anomaly detection tasks, we introduce existing IAD methods as vision experts and present a novel large multimodal model applying vision experts for industrial anomaly detection~(abbreviated to Myriad). Specifically, we utilize the anomaly map generated by the vision experts as guidance for LMMs, such that the vision model is guided to pay more attention to anomalous regions. Then, the visual features are modulated via an adapter to fit the anomaly detection tasks, which are fed into the language model together with the vision expert guidance and human instructions to generate the final outputs. Extensive experiments are applied on MVTec-AD, VisA, and PCB Bank benchmarks demonstrate that our proposed method not only performs favorably against state-of-the-art methods, but also inherits the flexibility and instruction-following ability of LMMs in the field of IAD. Source code and pre-trained models are publicly available at urlhttps://github.com/tzjtatata/Myriad.