# AA-CLIP: Enhancing Zero-shot Anomaly Detection via Anomaly-Aware CLIP

arXiv: https://arxiv.org/abs/2503.06661 · HTML: https://mlsanity.com/paper/2503.06661/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 64 references that are in the map.
- First posted on arXiv 9 Mar 2025.

## Builds on (most cited first, 30 of 51)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization](https://mlsanity.com/paper/2011.08785/) (2020)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [ClipCap: CLIP Prefix for Image Captioning](https://mlsanity.com/paper/2111.09734/) (2021)
- [Anomaly Detection via Reverse Distillation from One-Class Embedding](https://mlsanity.com/paper/2201.10703/) (2022)
- [SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation](https://mlsanity.com/paper/2207.14315/) (2022)
- [Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification](https://mlsanity.com/paper/2207.09519/) (2022)
- [CRIS: CLIP-Driven Referring Image Segmentation](https://mlsanity.com/paper/2111.15174/) (2021)
- [CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows](https://mlsanity.com/paper/2107.12571/) (2021)
- [WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation](https://mlsanity.com/paper/2303.14814/) (2023)
- [Side Adapter Network for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2302.12242/) (2023)
- [AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection](https://mlsanity.com/paper/2310.18961/) (2023)
- [A Unified Model for Multi-class Anomaly Detection](https://mlsanity.com/paper/2206.03687/) (2022)
- [SimpleNet: A Simple Network for Image Anomaly Detection and Localization](https://mlsanity.com/paper/2303.15140/) (2023)
- [CLIP2Video: Mastering Video-Text Retrieval via Image CLIP](https://mlsanity.com/paper/2106.11097/) (2021)
- [CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection](https://mlsanity.com/paper/2301.00785/) (2023)
- [VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization](https://mlsanity.com/paper/2104.10036/) (2021)
- [A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Images](https://mlsanity.com/paper/1612.00799/) (2016)
- [Teaching CLIP to Count to Ten](https://mlsanity.com/paper/2302.12066/) (2023)
- [ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentation](https://mlsanity.com/paper/2212.03588/) (2022)
- [DeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detection](https://mlsanity.com/paper/2211.11317/) (2022)
- [Student-Teacher Feature Pyramid Matching for Anomaly Detection](https://mlsanity.com/paper/2103.04257/) (2021)
- [APRIL-GAN: A Zero-/Few-Shot Anomaly Classification and Segmentation Method for CVPR 2023 VAND Workshop Challenge Tracks 1&2: 1st Place on Zero-shot AD and 4th Place on Few-shot AD](https://mlsanity.com/paper/2305.17382/) (2023)
- [Deep Learning for Medical Anomaly Detection -- A Survey](https://mlsanity.com/paper/2012.02364/) (2020)

## Built on by (most cited first, 30 of 73)

- [OmniAD: Detect and Understand Industrial Anomaly via Multimodal Reasoning](https://mlsanity.com/paper/2505.22039/) (2025)
- [Benchmarking Reward Hack Detection in Code Environments via Contrastive Analysis](https://mlsanity.com/paper/2601.20103/) (2026)
- [VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer](https://mlsanity.com/paper/2603.07952/) (2026)
- [AF-CLIP: Zero-Shot Anomaly Detection via Anomaly-Focused CLIP Adaptation](https://mlsanity.com/paper/2507.19949/) (2025)
- [MRAD: Zero-Shot Anomaly Detection with Memory-Driven Retrieval](https://mlsanity.com/paper/2602.00522/) (2026)
- [One Language-Free Foundation Model Is Enough for Universal Vision Anomaly Detection](https://mlsanity.com/paper/2601.05552/) (2026)
- [AnomalyVFM -- Transforming Vision Foundation Models into Zero-Shot Anomaly Detectors](https://mlsanity.com/paper/2601.20524/) (2026)
- [GS-CLIP: Zero-shot 3D Anomaly Detection by Geometry-Aware Prompt and Synergistic View Representation Learning](https://mlsanity.com/paper/2602.19206/) (2026)
- [Crane: Context-Guided Prompt Learning and Attention Refinement for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2504.11055/) (2025)
- [GenCLIP: Generalizing CLIP Prompts for Zero-shot Anomaly Detection](https://mlsanity.com/paper/2504.14919/) (2025)
- [Latent Anomaly Knowledge Excavation: Unveiling Sparse Sensitive Neurons in Vision-Language Models](https://mlsanity.com/paper/2604.07802/) (2026)
- [A Comprehensive Survey for Real-World Industrial Defect Detection: Challenges, Approaches, and Prospects](https://mlsanity.com/paper/2507.13378/) (2025)
- [Quality-Aware Language-Conditioned Local Auto-Regressive Anomaly Synthesis and Detection](https://mlsanity.com/paper/2508.03539/) (2025)
- [ADNet: A Large-Scale and Extensible Multi-Domain Benchmark for Anomaly Detection Across 380 Real-World Categories](https://mlsanity.com/paper/2511.20169/) (2025)
- [Back to Point: Exploring Point-Language Models for Zero-Shot 3D Anomaly Detection](https://mlsanity.com/paper/2603.21511/) (2026)
- [VitaTouch: Property-Aware Vision-Tactile-Language Model for Robotic Quality Inspection in Manufacturing](https://mlsanity.com/paper/2604.03322/) (2026)
- [IQE-CLIP: Instance-aware Query Embedding for Zero-/Few-shot Anomaly Detection in Medical Domain](https://mlsanity.com/paper/2506.10730/) (2025)
- [CoPS: Conditional Prompt Synthesis for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2508.03447/) (2025)
- [TRACES: Temporal Recall with Contextual Embeddings for Real-Time Video Anomaly Detection](https://mlsanity.com/paper/2511.00580/) (2025)
- [ABounD: Adversarial Boundary-Driven Few-Shot Learning for Multi-Class Anomaly Detection](https://mlsanity.com/paper/2511.22436/) (2025)
- [HAAF: Hierarchical Adaptation and Alignment of Foundation Models for Few-Shot Pathology Anomaly Detection](https://mlsanity.com/paper/2601.17405/) (2026)
- [DMP-3DAD: Cross-Category 3D Anomaly Detection via Realistic Depth Map Projection with Few Normal Samples](https://mlsanity.com/paper/2602.10806/) (2026)
- [AG-VAS: Anchor-Guided Zero-Shot Visual Anomaly Segmentation with Large Multimodal Models](https://mlsanity.com/paper/2603.01305/) (2026)
- [WMoE-CLIP: Wavelet-Enhanced Mixture-of-Experts Prompt Learning for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2603.06313/) (2026)
- [FB-CLIP: Fine-Grained Zero-Shot Anomaly Detection with Foreground-Background Disentanglement](https://mlsanity.com/paper/2603.19608/) (2026)
- [SPG: Sparse-Projected Guides with Sparse Autoencoders for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2604.02871/) (2026)
- [PASTA: Vision Transformer Patch Aggregation for Weakly Supervised Target and Anomaly Segmentation](https://mlsanity.com/paper/2604.09701/) (2026)
- [AnomalyAgent: Training-Free Agentic Models for Zero-/Few-Shot Anomaly Detection](https://mlsanity.com/paper/2605.30140/) (2026)
- [SD-MAD: Sign-Driven Few-shot Multi-Anomaly Detection in Medical Images](https://mlsanity.com/paper/2505.16659/) (2025)
- [ViP2-CLIP: Visual-Perception Prompting with Unified Alignment for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2505.17692/) (2025)

## Abstract (arXiv)

Anomaly detection (AD) identifies outliers for applications like defect and lesion detection. While CLIP shows promise for zero-shot AD tasks due to its strong generalization capabilities, its inherent Anomaly-Unawareness leads to limited discrimination between normal and abnormal features. To address this problem, we propose Anomaly-Aware CLIP (AA-CLIP), which enhances CLIP's anomaly discrimination ability in both text and visual spaces while preserving its generalization capability. AA-CLIP is achieved through a straightforward yet effective two-stage approach: it first creates anomaly-aware text anchors to differentiate normal and abnormal semantics clearly, then aligns patch-level visual features with these anchors for precise anomaly localization. This two-stage strategy, with the help of residual adapters, gradually adapts CLIP in a controlled manner, achieving effective AD while maintaining CLIP's class knowledge. Extensive experiments validate AA-CLIP as a resource-efficient solution for zero-shot AD tasks, achieving state-of-the-art results in industrial and medical applications. The code is available at https://github.com/Mwxinnn/AA-CLIP.