mlsanityData to 7 Oct 2026

Anomaly Detection by Adapting a pre-trained Vision Language Model

arXiv 2403.09493 ↗First posted 14 Mar 2024cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

Recently, large vision and language models have shown their success when adapting them to many downstream tasks. In this paper, we present a unified framework named CLIP-ADA for Anomaly Detection by Adapting a pre-trained CLIP model. To this end, we make two important improvements: 1) To acquire unified anomaly detection across industrial images of multiple categories, we introduce the learnable prompt and propose to associate it with abnormal patterns through self-supervised learning. 2) To fully exploit the representation power of CLIP, we introduce an anomaly region refinement strategy to refine the localization quality. During testing, the anomalies are localized by directly calculating the similarity between the representation of the learnable prompt and the image. Comprehensive experiments demonstrate the superiority of our framework, e.g., we achieve the state-of-the-art 97.5/55.6 and 89.3/33.1 on MVTec-AD and VisA for anomaly detection and localization. In addition, the proposed method also achieves encouraging performance with marginal training data, which is more challenging.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 27 of its 32 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

27 of 32 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  6. 2021Towards Total Recall in Industrial Anomaly Detectioncited by 772
  7. 2020PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localizationcited by 591
  8. 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 530
  9. 2021CutPaste: Self-Supervised Learning for Anomaly Detection and Localizationcited by 529
  10. 2021DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detectioncited by 510
  11. 2022Anomaly Detection via Reverse Distillation from One-Class Embeddingcited by 419
  12. 2022SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentationcited by 397
15 more
  1. 2023WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentationcited by 311
  2. 2022Generalized Decoding for Pixel, Image, and Languagecited by 292
  3. 2020Multiresolution Knowledge Distillation for Anomaly Detectioncited by 285
  4. 2023AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detectioncited by 270
  5. 2022A Unified Model for Multi-class Anomaly Detectioncited by 267
  6. 2021A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Modelcited by 223
  7. 2020PANDA: Adapting Pretrained Features for Anomaly Detection and Segmentationcited by 175
  8. 2023APRIL-GAN: A Zero-/Few-Shot Anomaly Classification and Segmentation Method for CVPR 2023 VAND Workshop Challenge Tracks 1&2: 1st Place on Zero-shot AD and 4th Place on Few-shot ADcited by 128
  9. 2021Natural Synthetic Anomalies for Self-Supervised Anomaly Detection and Localizationcited by 116
  10. 2019Towards Visually Explaining Variational Autoencoderscited by 113
  11. 2022Exploiting Unlabeled Data with Vision and Language Models for Object Detectioncited by 97
  12. 2022DSR -- A dual subspace re-projection network for surface anomaly detectioncited by 89
  13. 2022ProposalCLIP: Unsupervised Open-Category Object Proposal Generation via Exploiting CLIP Cuescited by 42
  14. 2023DiAD: A Diffusion-based Framework for Multi-class Anomaly Detectioncited by 33
  15. 2022Reconstruction from edge image combined with color and gradient difference for industrial surface anomaly detectioncited by 18

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024VL4AD: Vision-Language Models Improve Pixel-wise Anomaly Detectioncited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2025AdaptCLIP: Adapting CLIP for Universal Visual Anomaly Detection
  2. 2024AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detection
  3. 2024Dual-Image Enhanced CLIP for Zero-Shot Anomaly Detection
  4. 2024FADE: Few-shot/zero-shot Anomaly Detection Engine using Large Vision-Language Model
  5. 2024SOWA: Adapting Hierarchical Frozen Window Self-Attention to Visual-Language Models for Better Anomaly Detection
  6. 2023AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models
  7. 2024PromptAD: Learning Prompts with only Normal Samples for Few-Shot Anomaly Detection
  8. 2024Do LLMs Understand Visual Anomalies? Uncovering LLM's Capabilities in Zero-shot Anomaly Detection