mlsanityData to 7 Oct 2026

AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models

arXiv 2308.15366 ↗First posted 29 Aug 2023cs.CVcited by 169

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

Large Vision-Language Models (LVLMs) such as MiniGPT-4 and LLaVA have demonstrated the capability of understanding images and achieved remarkable performance in various visual tasks. Despite their strong abilities in recognizing common objects due to extensive training datasets, they lack specific domain knowledge and have a weaker understanding of localized details within objects, which hinders their effectiveness in the Industrial Anomaly Detection (IAD) task. On the other hand, most existing IAD methods only provide anomaly scores and necessitate the manual setting of thresholds to distinguish between normal and abnormal samples, which restricts their practical implementation. In this paper, we explore the utilization of LVLM to address the IAD problem and propose AnomalyGPT, a novel IAD approach based on LVLM. We generate training data by simulating anomalous images and producing corresponding textual descriptions for each image. We also employ an image decoder to provide fine-grained semantic and design a prompt learner to fine-tune the LVLM using prompt embeddings. Our AnomalyGPT eliminates the need for manual threshold adjustments, thus directly assesses the presence and locations of anomalies. Additionally, AnomalyGPT supports multi-turn dialogues and exhibits impressive few-shot in-context learning capabilities. With only one normal shot, AnomalyGPT achieves the state-of-the-art performance with an accuracy of 86.1%, an image-level AUC of 94.1%, and a pixel-level AUC of 95.3% on the MVTec-AD dataset. Code is available at https://github.com/CASIA-IVA-Lab/AnomalyGPT.

Key facts. Cited by 169 papers in the map (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 42 references that are in the map. First posted on arXiv 29 Aug 2023.

Builds on

29 of 42 references.

References · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  8. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  9. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  10. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  11. 2021Towards Total Recall in Industrial Anomaly Detectioncited by 772
  12. 2017Focal Loss for Dense Object Detectioncited by 749
17 more
  1. 2020PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localizationcited by 591
  2. 2021CutPaste: Self-Supervised Learning for Anomaly Detection and Localizationcited by 529
  3. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
  4. 2022SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentationcited by 397
  5. 2023PandaGPT: One Model To Instruction-Follow Them Allcited by 330
  6. 2021CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flowscited by 320
  7. 2023WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentationcited by 311
  8. 2020Sub-Image Anomaly Detection with Deep Pyramid Correspondencescited by 297
  9. 2022A Unified Model for Multi-class Anomaly Detectioncited by 267
  10. 2020Patch SVDD: Patch-level SVDD for Anomaly Detection and Segmentationcited by 212
  11. 2022CFA: Coupled-hypersphere-based Feature Adaptation for Target-Oriented Anomaly Localizationcited by 135
  12. 2022Registration based Few-Shot Anomaly Detectioncited by 133
  13. 2023APRIL-GAN: A Zero-/Few-Shot Anomaly Classification and Segmentation Method for CVPR 2023 VAND Workshop Challenge Tracks 1&2: 1st Place on Zero-shot AD and 4th Place on Few-shot ADcited by 128
  14. 2021Natural Synthetic Anomalies for Self-Supervised Anomaly Detection and Localizationcited by 116
  15. 2021Inpainting Transformer for Anomaly Detectioncited by 91
  16. 2023Pushing the Limits of Fewshot Anomaly Detection in Industry Vision: Graphcorecited by 75
  17. 2023PyramidFlow: High-Resolution Defect Contrastive Localization using Pyramid Normalizing Flowcited by 74

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 41; 2025: 71; 2026: 51.

169 papers.

Sort:

By area:

  1. 2024AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detectioncited by 126
  2. 2024SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Modelcited by 107
  3. 2024Harnessing Large Language Models for Training-free Video Anomaly Detectioncited by 85
  4. 2024Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLMcited by 63
  5. 2024Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Modelscited by 55
  6. 2025Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Modelscited by 54
  7. 2024FiLo: Zero-Shot Anomaly Detection by Fine-Grained Description and High-Quality Localizationcited by 51
  8. 2024VCP-CLIP: A visual context prompting model for zero-shot anomaly segmentationcited by 49
  9. 2024Advancing Video Anomaly Detection: A Concise Review and a New Datasetcited by 48
  10. 2024Position: What Can Large Language Models Tell Us about Time Series Analysiscited by 47
  11. 2025Exploring Intrinsic Normal Prototypes within a Single Image for Universal Anomaly Detectioncited by 47
  12. 2023Towards Generic Anomaly Detection and Understanding: Large-scale Visual-linguistic Model (GPT-4V) Takes the Leadcited by 40
38 more
  1. 2023Myriad: Large Multimodal Model by Applying Vision Experts for Industrial Anomaly Detectioncited by 39
  2. 2024Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Surveycited by 36
  3. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  4. 2024Effectiveness Assessment of Recent Large Vision-Language Modelscited by 29
  5. 2024GEMeX: A Large-Scale, Groundable, and Explainable Medical VQA Benchmark for Chest X-ray Diagnosiscited by 28
  6. 2024UniVAD: A Training-free Unified Model for Few-shot Visual Anomaly Detectioncited by 28
  7. 2025The MVTec AD 2 Dataset: Advanced Scenarios for Unsupervised Anomaly Detectioncited by 28
  8. 2025AnomalyR1: A GRPO-based End-to-end MLLM for Industrial Anomaly Detectioncited by 28
  9. 2025Towards Training-free Anomaly Detection with Vision and Language Foundation Modelscited by 24
  10. 2023GPT-4V-AD: Exploring Grounding Potential of VQA-oriented GPT-4V for Zero-shot Anomaly Detectioncited by 23
  11. 2024Unseen Visual Anomaly Generationcited by 23
  12. 2024Fine-grained Abnormality Prompt Learning for Zero-shot Anomaly Detectioncited by 22
  13. 2024PixOOD: Pixel-Level Out-of-Distribution Detectioncited by 20
  14. 2025Unlocking the Capabilities of Large Vision-Language Models for Generalizable and Explainable Deepfake Detectioncited by 20
  15. 2024LogiCode: an LLM-Driven Framework for Logical Anomaly Detectioncited by 18
  16. 2024LLM-3D Print: Large Language Models To Monitor and Control 3D Printingcited by 18
  17. 2025Learning to Detect Multi-class Anomalies with Just One Normal Image Promptcited by 18
  18. 2024Long-Tailed Anomaly Detection with Learnable Class Namescited by 14
  19. 2025MM-Skin: Enhancing Dermatology Vision-Language Model with an Image-Text Dataset Derived from Textbookscited by 14
  20. 2025Object Detection with Multimodal Large Vision-Language Models: An In-depth Reviewcited by 14
  21. 2024VMAD: Visual-enhanced Multimodal Large Language Model for Zero-Shot Anomaly Detectioncited by 13
  22. 2024PrefixKV: Adaptive Prefix KV Cache is What Vision Instruction-Following Models Need for Efficient Generationcited by 13
  23. 2024FabGPT: An Efficient Large Multimodal Model for Complex Wafer Defect Knowledge Queriescited by 12
  24. 2025MultiADS: Defect-aware Supervision for Multi-type Anomaly Detection and Segmentation in Zero-Shot Learningcited by 12
  25. 2023VQA4CIR: Boosting Composed Image Retrieval with Visual Question Answeringcited by 11
  26. 2024ForgeryGPT: A Multimodal LLM for Interpretable Image Forgery Detection and Localizationcited by 11
  27. 2025EIAD: Explainable Industrial Anomaly Detection Via Multi-Modal Large Language Modelscited by 11
  28. 2023Bootstrap Fine-Grained Vision-Language Alignment for Unified Zero-Shot Anomaly Localizationcited by 10
  29. 2025DictAS: A Framework for Class-Generalizable Few-Shot Anomaly Segmentation via Dictionary Lookupcited by 9
  30. 2025Triad: Empowering LMM-based Anomaly Detection with Vision Expert-guided Visual Tokenizer and Manufacturing Processcited by 8
  31. 2025Adaptive Multi-prompt Contrastive Network for Few-shot Out-of-distribution Detectioncited by 8
  32. 2025AnomalyMoE: Towards a Language-free Generalist Model for Unified Visual Anomaly Detectioncited by 8
  33. 2025A Neurosymbolic Framework for Interpretable Cognitive Attack Detection in Augmented Realitycited by 8
  34. 2026From Zero to Hero: Advancing Zero-Shot Foundation Models for Tabular Outlier Detectioncited by 8
  35. 2024Dual-path Frequency Discriminators for Few-shot Anomaly Detectioncited by 7
  36. 2024FADE: Few-shot/zero-shot Anomaly Detection Engine using Large Vision-Language Modelcited by 7
  37. 2025FiLo++: Zero-/Few-Shot Anomaly Detection by Fused Fine-Grained Descriptions and Deformable Localizationcited by 7
  38. 2025One-to-Normal: Anomaly Personalization for Few-shot Anomaly Detectioncited by 7

50 most cited of 169 All 169 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025IADGPT: Unified LVLM for Few-Shot Industrial Anomaly Detection, Localization, and Reasoning via In-Context Learning
  2. 2026InspectorGPT: A Comparative Reasoning Enhanced VLM for Comprehensive Industrial Anomaly Detection
  3. 2024MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection
  4. 2024Customizing Visual-Language Foundation Models for Multi-modal Anomaly Detection and Reasoning
  5. 2026ZSG-IAD: A Multimodal Framework for Zero-Shot Grounded Industrial Anomaly Detection
  6. 2025LR-IAD:Mask-Free Industrial Anomaly Detection with Logical Reasoning
  7. 2025IAD-R1: Reinforcing Consistent Reasoning in Industrial Anomaly Detection
  8. 2024Do LLMs Understand Visual Anomalies? Uncovering LLM's Capabilities in Zero-shot Anomaly Detection