# AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detection

arXiv: https://arxiv.org/abs/2407.15795 · HTML: https://mlsanity.com/paper/2407.15795/

## Key facts

- Cited by 127 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 57 references that are in the map.
- First posted on arXiv 22 Jul 2024.

## Builds on (most cited first, 30 of 38)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [MaPLe: Multi-modal Prompt Learning](https://mlsanity.com/paper/2210.03117/) (2022)
- [Anomaly Detection via Reverse Distillation from One-Class Embedding](https://mlsanity.com/paper/2201.10703/) (2022)
- [SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation](https://mlsanity.com/paper/2207.14315/) (2022)
- [WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation](https://mlsanity.com/paper/2303.14814/) (2023)
- [Deep Semi-Supervised Anomaly Detection](https://mlsanity.com/paper/1906.02694/) (2019)
- [AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection](https://mlsanity.com/paper/2310.18961/) (2023)
- [VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization](https://mlsanity.com/paper/2104.10036/) (2021)
- [Skin Lesion Analysis toward Melanoma Detection: A Challenge at the International Symposium on Biomedical Imaging (ISBI) 2016, hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1605.01397/) (2016)
- [AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models](https://mlsanity.com/paper/2308.15366/) (2023)
- [Segmentation-Based Deep-Learning Approach for Surface-Defect Detection](https://mlsanity.com/paper/1903.08536/) (2019)
- [Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detection](https://mlsanity.com/paper/2403.12580/) (2024)
- [Registration based Few-Shot Anomaly Detection](https://mlsanity.com/paper/2207.07361/) (2022)
- [APRIL-GAN: A Zero-/Few-Shot Anomaly Classification and Segmentation Method for CVPR 2023 VAND Workshop Challenge Tracks 1&2: 1st Place on Zero-shot AD and 4th Place on Few-shot AD](https://mlsanity.com/paper/2305.17382/) (2023)
- [Catching Both Gray and Black Swans: Open-set Supervised Anomaly Detection](https://mlsanity.com/paper/2203.14506/) (2022)
- [DSR -- A dual subspace re-projection network for surface anomaly detection](https://mlsanity.com/paper/2208.01521/) (2022)
- [Real3D-AD: A Dataset of Point Cloud Anomaly Detection](https://mlsanity.com/paper/2309.13226/) (2023)
- [Deep Learning for Chest X-ray Analysis: A Survey](https://mlsanity.com/paper/2103.08700/) (2021)
- [A Survey on Visual Anomaly Detection: Challenge, Approach, and Prospect](https://mlsanity.com/paper/2401.16402/) (2024)
- [Instance-aware Dynamic Prompt Tuning for Pre-trained Point Cloud Models](https://mlsanity.com/paper/2304.07221/) (2023)
- [Segment Any Anomaly without Training via Hybrid Prompt Regularization](https://mlsanity.com/paper/2305.10724/) (2023)

## Built on by (most cited first, 30 of 127)

- [AA-CLIP: Enhancing Zero-shot Anomaly Detection via Anomaly-Aware CLIP](https://mlsanity.com/paper/2503.06661/) (2025)
- [Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Models](https://mlsanity.com/paper/2502.07601/) (2025)
- [Exploring Intrinsic Normal Prototypes within a Single Image for Universal Anomaly Detection](https://mlsanity.com/paper/2503.02424/) (2025)
- [Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey](https://mlsanity.com/paper/2407.21794/) (2024)
- [Bayesian Prompt Flow Learning for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2503.10080/) (2025)
- [AdaptCLIP: Adapting CLIP for Universal Visual Anomaly Detection](https://mlsanity.com/paper/2505.09926/) (2025)
- [Fine-grained Abnormality Prompt Learning for Zero-shot Anomaly Detection](https://mlsanity.com/paper/2410.10289/) (2024)
- [Towards Zero-shot Point Cloud Anomaly Detection: A Multi-View Projection Framework](https://mlsanity.com/paper/2409.13162/) (2024)
- [Towards High-Resolution 3D Anomaly Detection: A Scalable Dataset and Real-Time Framework for Subtle Industrial Defects](https://mlsanity.com/paper/2507.07435/) (2025)
- [MultiADS: Defect-aware Supervision for Multi-type Anomaly Detection and Segmentation in Zero-Shot Learning](https://mlsanity.com/paper/2504.06740/) (2025)
- [VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer](https://mlsanity.com/paper/2603.07952/) (2026)
- [DictAS: A Framework for Class-Generalizable Few-Shot Anomaly Segmentation via Dictionary Lookup](https://mlsanity.com/paper/2508.13560/) (2025)
- [Normal-Abnormal Guided Generalist Anomaly Detection](https://mlsanity.com/paper/2510.00495/) (2025)
- [Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Review](https://mlsanity.com/paper/2506.18378/) (2025)
- [Triad: Empowering LMM-based Anomaly Detection with Vision Expert-guided Visual Tokenizer and Manufacturing Process](https://mlsanity.com/paper/2503.13184/) (2025)
- [Track Any Anomalous Object: A Granular Video Anomaly Detection Pipeline](https://mlsanity.com/paper/2506.05175/) (2025)
- [Foundation Models and Transformers for Anomaly Detection: A Survey](https://mlsanity.com/paper/2507.15905/) (2025)
- [No Need For Real Anomaly: MLLM Empowered Zero-Shot Video Anomaly Detection](https://mlsanity.com/paper/2602.19248/) (2026)
- [Deep learning based infrared small object segmentation: Challenges and future directions](https://mlsanity.com/paper/2502.14168/) (2025)
- [INP-Former++: Advancing Universal Anomaly Detection via Intrinsic Normal Prototypes and Residual Learning](https://mlsanity.com/paper/2506.03660/) (2025)
- [AF-CLIP: Zero-Shot Anomaly Detection via Anomaly-Focused CLIP Adaptation](https://mlsanity.com/paper/2507.19949/) (2025)
- [SubspaceAD: Training-Free Few-Shot Anomaly Detection via Subspace Modeling](https://mlsanity.com/paper/2602.23013/) (2026)
- [MRAD: Zero-Shot Anomaly Detection with Memory-Driven Retrieval](https://mlsanity.com/paper/2602.00522/) (2026)
- [RAD: A Dataset and Benchmark for Real-Life Anomaly Detection with Robotic Observations](https://mlsanity.com/paper/2410.00713/) (2024)
- [A Survey on Foundation-Model-Based Industrial Defect Detection](https://mlsanity.com/paper/2502.19106/) (2025)
- [AnomalyPainter: Vision-Language-Diffusion Synergy for Zero-Shot Realistic and Diverse Industrial Anomaly Synthesis](https://mlsanity.com/paper/2503.07253/) (2025)
- [UltraAD: Fine-Grained Ultrasound Anomaly Classification via Few-Shot CLIP Adaptation](https://mlsanity.com/paper/2506.19694/) (2025)
- [A Lightweight 3D Anomaly Detection Method with Rotationally Invariant Features](https://mlsanity.com/paper/2511.13115/) (2025)
- [AD-Copilot: A Vision-Language Assistant for Industrial Anomaly Detection via Visual In-context Comparison](https://mlsanity.com/paper/2603.13779/) (2026)
- [Exploiting Point-Language Models with Dual-Prompts for 3D Anomaly Detection](https://mlsanity.com/paper/2502.11307/) (2025)

## Abstract (arXiv)

Zero-shot anomaly detection (ZSAD) targets the identification of anomalies within images from arbitrary novel categories. This study introduces AdaCLIP for the ZSAD task, leveraging a pre-trained vision-language model (VLM), CLIP. AdaCLIP incorporates learnable prompts into CLIP and optimizes them through training on auxiliary annotated anomaly detection data. Two types of learnable prompts are proposed: static and dynamic. Static prompts are shared across all images, serving to preliminarily adapt CLIP for ZSAD. In contrast, dynamic prompts are generated for each test image, providing CLIP with dynamic adaptation capabilities. The combination of static and dynamic prompts is referred to as hybrid prompts, and yields enhanced ZSAD performance. Extensive experiments conducted across 14 real-world anomaly detection datasets from industrial and medical domains indicate that AdaCLIP outperforms other ZSAD methods and can generalize better to different categories and even domains. Finally, our analysis highlights the importance of diverse auxiliary data and optimized prompts for enhanced generalization capacity. Code is available at https://github.com/caoyunkang/AdaCLIP.