# AdaptCLIP: Adapting CLIP for Universal Visual Anomaly Detection

arXiv: https://arxiv.org/abs/2505.09926 · HTML: https://mlsanity.com/paper/2505.09926/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 56 references that are in the map.
- First posted on arXiv 15 May 2025.

## Builds on (most cited first, 30 of 42)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization](https://mlsanity.com/paper/2011.08785/) (2020)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [CutPaste: Self-Supervised Learning for Anomaly Detection and Localization](https://mlsanity.com/paper/2104.04015/) (2021)
- [DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detection](https://mlsanity.com/paper/2108.07610/) (2021)
- [Anomaly Detection via Reverse Distillation from One-Class Embedding](https://mlsanity.com/paper/2201.10703/) (2022)
- [SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation](https://mlsanity.com/paper/2207.14315/) (2022)
- [Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings](https://mlsanity.com/paper/1911.02357/) (2019)
- [WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation](https://mlsanity.com/paper/2303.14814/) (2023)
- [Multiresolution Knowledge Distillation for Anomaly Detection](https://mlsanity.com/paper/2011.11108/) (2020)
- [AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection](https://mlsanity.com/paper/2310.18961/) (2023)
- [A Unified Model for Multi-class Anomaly Detection](https://mlsanity.com/paper/2206.03687/) (2022)
- [SimpleNet: A Simple Network for Image Anomaly Detection and Localization](https://mlsanity.com/paper/2303.15140/) (2023)
- [VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization](https://mlsanity.com/paper/2104.10036/) (2021)
- [A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Images](https://mlsanity.com/paper/1612.00799/) (2016)
- [OCGAN: One-class Novelty Detection Using GANs with Constrained Latent Representations](https://mlsanity.com/paper/1903.08550/) (2019)
- [The MVTec 3D-AD Dataset for Unsupervised 3D Anomaly Detection and Localization](https://mlsanity.com/paper/2112.09045/) (2021)
- [Student-Teacher Feature Pyramid Matching for Anomaly Detection](https://mlsanity.com/paper/2103.04257/) (2021)
- [Segmentation-Based Deep-Learning Approach for Surface-Defect Detection](https://mlsanity.com/paper/1903.08536/) (2019)
- [Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detection](https://mlsanity.com/paper/2403.12580/) (2024)
- [Registration based Few-Shot Anomaly Detection](https://mlsanity.com/paper/2207.07361/) (2022)
- [AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2407.15795/) (2024)
- [Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection](https://mlsanity.com/paper/2111.09099/) (2021)
- [Fully Convolutional Cross-Scale-Flows for Image-based Defect Detection](https://mlsanity.com/paper/2110.02855/) (2021)
- [Old is Gold: Redefining the Adversarially Learned One-Class Classifier Training Paradigm](https://mlsanity.com/paper/2004.07657/) (2020)
- [Modeling the Distribution of Normal Data in Pre-Trained Deep Features for Anomaly Detection](https://mlsanity.com/paper/2005.14140/) (2020)
- [Generalized Denoising Auto-Encoders as Generative Models](https://mlsanity.com/paper/1305.6663/) (2013)
- [Catching Both Gray and Black Swans: Open-set Supervised Anomaly Detection](https://mlsanity.com/paper/2203.14506/) (2022)

## Built on by (most cited first, 29 of 29)

- [VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer](https://mlsanity.com/paper/2603.07952/) (2026)
- [Foundation Visual Encoders Are Secretly Few-Shot Anomaly Detectors](https://mlsanity.com/paper/2510.01934/) (2025)
- [Towards Fine-Grained Vision-Language Alignment for Few-Shot Anomaly Detection](https://mlsanity.com/paper/2510.26464/) (2025)
- [One Language-Free Foundation Model Is Enough for Universal Vision Anomaly Detection](https://mlsanity.com/paper/2601.05552/) (2026)
- [Latent Anomaly Knowledge Excavation: Unveiling Sparse Sensitive Neurons in Vision-Language Models](https://mlsanity.com/paper/2604.07802/) (2026)
- [Real-IAD Variety: Pushing Industrial Anomaly Detection Dataset to a Modern Era](https://mlsanity.com/paper/2511.00540/) (2025)
- [CoPS: Conditional Prompt Synthesis for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2508.03447/) (2025)
- [TRACES: Temporal Recall with Contextual Embeddings for Real-Time Video Anomaly Detection](https://mlsanity.com/paper/2511.00580/) (2025)
- [ABounD: Adversarial Boundary-Driven Few-Shot Learning for Multi-Class Anomaly Detection](https://mlsanity.com/paper/2511.22436/) (2025)
- [HAAF: Hierarchical Adaptation and Alignment of Foundation Models for Few-Shot Pathology Anomaly Detection](https://mlsanity.com/paper/2601.17405/) (2026)
- [M3-AD: Reflection-aware Multi-modal, Multi-category, and Multi-dimensional Benchmark and Framework for Industrial Anomaly Detection](https://mlsanity.com/paper/2603.00055/) (2026)
- [FB-CLIP: Fine-Grained Zero-Shot Anomaly Detection with Foreground-Background Disentanglement](https://mlsanity.com/paper/2603.19608/) (2026)
- [SPG: Sparse-Projected Guides with Sparse Autoencoders for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2604.02871/) (2026)
- [Quantized FCA: Efficient Zero-Shot Texture Anomaly Detection](https://mlsanity.com/paper/2510.15602/) (2025)
- [TokenCLIP: Token-wise Prompt Learning for Zero-shot Anomaly Detection](https://mlsanity.com/paper/2510.21171/) (2025)
- [Large-Scale Universal Defect Generation: Foundation Models and Datasets](https://mlsanity.com/paper/2604.08915/) (2026)
- [H2VLR: Heterogeneous Hypergraph Vision-Language Reasoning for Few-Shot Anomaly Detection](https://mlsanity.com/paper/2604.14507/) (2026)
- [Res2CLIP: Few-Shot Generalist Anomaly Detection with Residual-to-Residual Alignment](https://mlsanity.com/paper/2605.16171/) (2026)
- [EntroAD: Structural Entropy-Guided Prompt Adaptation for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2605.28630/) (2026)
- [Value-order Decomposition for Generalist Anomaly Detection](https://mlsanity.com/paper/2606.14475/) (2026)
- [EdgeZSAD: Practical Zero-Shot Anomaly Detection on Edge Devices](https://mlsanity.com/paper/2606.16119/) (2026)
- [CoGeoAD: Hierarchical Color-Geometric Fusion with Multi-View Attention for Zero-Shot 3D Anomaly Detection](https://mlsanity.com/paper/2606.25273/) (2026)
- [Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detection](https://mlsanity.com/paper/2607.03817/) (2026)
- [DuoAD: Leveraging [CLS] Dual Characteristics for Training-Free Few-Shot Anomaly Detection](https://mlsanity.com/paper/2607.23924/) (2026)
- [VFAD: Variational Semantic Prompting Meets Frequency-Adaptive Representation Learning for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2607.29370/) (2026)
- [Co-Evolutionary Prompt Optimization with Cross-Category Transfer for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2608.29467/) (2026)
- [Exploring the Potential of Contrastive Language-Image Pre-training for Multi-Source Remote Sensing Data](https://mlsanity.com/paper/2609.03391/) (2026)
- [PL-SCEA: Reconfiguring Pretrained Attention for Few-Shot Industrial Anomaly Detection](https://mlsanity.com/paper/2609.03655/) (2026)
- [TED:Text-Axis Evidence Decomposition for Prompted Anomaly Localization](https://mlsanity.com/paper/2609.39033/) (2026)

## Abstract (arXiv)

Universal visual anomaly detection aims to identify anomalies from novel or unseen vision domains without additional fine-tuning, which is critical in open scenarios. Recent studies have demonstrated that pre-trained vision-language models like CLIP exhibit strong generalization with just zero or a few normal images. However, existing methods struggle with designing prompt templates, complex token interactions, or requiring additional fine-tuning, resulting in limited flexibility. In this work, we present a simple yet effective method called AdaptCLIP based on two key insights. First, adaptive visual and textual representations should be learned alternately rather than jointly. Second, comparative learning between query and normal image prompt should incorporate both contextual and aligned residual features, rather than relying solely on residual features. AdaptCLIP treats CLIP models as a foundational service, adding only three simple adapters, visual adapter, textual adapter, and prompt-query adapter, at its input or output ends. AdaptCLIP supports zero-/few-shot generalization across domains and possesses a training-free manner on target domains once trained on a base dataset. AdaptCLIP achieves state-of-the-art performance on 12 anomaly detection benchmarks from industrial and medical domains, significantly outperforming existing competitive methods. We will make the code and model of AdaptCLIP available at https://github.com/gaobb/AdaptCLIP.