mlsanityData to 14 Oct 2026

KAnoCLIP: Zero-Shot Anomaly Detection through Knowledge-Driven Prompt Learning and Enhanced Cross-Modal Integration

arXiv 2501.03786 ↗First posted 7 Jan 2025cs.CVcited by 7

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

Zero-shot anomaly detection (ZSAD) identifies anomalies without needing training samples from the target dataset, essential for scenarios with privacy concerns or limited data. Vision-language models like CLIP show potential in ZSAD but have limitations: relying on manually crafted fixed textual descriptions or anomaly prompts is time-consuming and prone to semantic ambiguity, and CLIP struggles with pixel-level anomaly segmentation, focusing more on global semantics than local details. To address these limitations, We introduce KAnoCLIP, a novel ZSAD framework that leverages vision-language models. KAnoCLIP combines general knowledge from a Large Language Model (GPT-3.5) and fine-grained, image-specific knowledge from a Visual Question Answering system (Llama3) via Knowledge-Driven Prompt Learning (KnPL). KnPL uses a knowledge-driven (KD) loss function to create learnable anomaly prompts, removing the need for fixed text prompts and enhancing generalization. KAnoCLIP includes the CLIP visual encoder with V-V attention (CLIP-VV), Bi-Directional Cross-Attention for Multi-Level Cross-Modal Interaction (Bi-CMCI), and Conv-Adapter. These components preserve local visual semantics, improve local cross-modal fusion, and align global visual features with textual information, enhancing pixel-level anomaly detection. KAnoCLIP achieves state-of-the-art performance in ZSAD across 12 industrial and medical datasets, demonstrating superior generalization compared to existing methods.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 17 of its 26 references that are in the map. First posted on arXiv 7 Jan 2025.

Builds on

17 of 26 references.

References · 17

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  3. 2021Towards Total Recall in Industrial Anomaly Detectioncited by 776
  4. 2019The Liver Tumor Segmentation Benchmark (LiTS)cited by 509
  5. 2022SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentationcited by 400
  6. 2023WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentationcited by 315
  7. 2023Visual-Language Prompt Tuning with Knowledge-guided Context Optimizationcited by 308
  8. 2022Vision Transformer with Deformable Attentioncited by 295
  9. 2020Multiresolution Knowledge Distillation for Anomaly Detectioncited by 285
  10. 2023AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detectioncited by 271
  11. 2021VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localizationcited by 223
  12. 2019Cross Attention Network for Few-shot Classificationcited by 219
5 more
  1. 2019Dice Loss for Data-imbalanced NLP Taskscited by 182
  2. 2019Segmentation-Based Deep-Learning Approach for Surface-Defect Detectioncited by 145
  3. 2023APRIL-GAN: A Zero-/Few-Shot Anomaly Classification and Segmentation Method for CVPR 2023 VAND Workshop Challenge Tracks 1&2: 1st Place on Zero-shot AD and 4th Place on Few-shot ADcited by 130
  4. 2024Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Imagescited by 69
  5. 2021Self-supervised Pseudo Multi-class Pre-training for Unsupervised Anomaly Detection and Segmentation in Medical Imagescited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025A Survey on Foundation-Model-Based Industrial Defect Detectioncited by 5
  2. 2026GS-CLIP: Zero-shot 3D Anomaly Detection by Geometry-Aware Prompt and Synergistic View Representation Learningcited by 4
  3. 2025Multimodal RAG-driven Anomaly Detection and Classification in Laser Powder Bed Fusion using Large Language Modelscited by 3
  4. 2025TRACES: Temporal Recall with Contextual Embeddings for Real-Time Video Anomaly Detectioncited by 1
  5. 2025ViP2-CLIP: Visual-Perception Prompting with Unified Alignment for Zero-Shot Anomaly Detectioncited by 0
  6. 2026HarmoniAD: Harmonizing Local Structures and Global Semantics for Anomaly Detectioncited by 0
  7. 2026Bridging Vision Foundation Model Priors with CLIP for Spatial-aware Few-shot Anomaly Detection in Medical Imagescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detection
  2. 2025PromptMoE: Generalizable Zero-Shot Anomaly Detection via Visually-Guided Prompt Mixtures
  3. 2026VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer
  4. 2026SSVP: Synergistic Semantic-Visual Prompting for Industrial Zero-Shot Anomaly Detection
  5. 2024VCP-CLIP: A visual context prompting model for zero-shot anomaly segmentation
  6. 2025Defect-aware Hybrid Prompt Optimization via Progressive Tuning for Zero-Shot Multi-type Anomaly Detection and Segmentation
  7. 2026Proximity-CLIP: Text-Guided Semantic Proximity Learning for Zero-Shot Anomaly Detection
  8. 2023CLIP-AD: A Language-Guided Staged Dual-Path Model for Zero-shot Anomaly Detection