# VCP-CLIP: A visual context prompting model for zero-shot anomaly segmentation

arXiv: https://arxiv.org/abs/2407.12276 · HTML: https://mlsanity.com/paper/2407.12276/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 25 of its 37 references that are in the map.
- First posted on arXiv 17 Jul 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting](https://mlsanity.com/paper/2112.01518/) (2021)
- [SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation](https://mlsanity.com/paper/2207.14315/) (2022)
- [Faster Segment Anything: Towards Lightweight SAM for Mobile Applications](https://mlsanity.com/paper/2306.14289/) (2023)
- [WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation](https://mlsanity.com/paper/2303.14814/) (2023)
- [AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection](https://mlsanity.com/paper/2310.18961/) (2023)
- [VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization](https://mlsanity.com/paper/2104.10036/) (2021)
- [ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentation](https://mlsanity.com/paper/2212.03588/) (2022)
- [AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models](https://mlsanity.com/paper/2308.15366/) (2023)
- [APRIL-GAN: A Zero-/Few-Shot Anomaly Classification and Segmentation Method for CVPR 2023 VAND Workshop Challenge Tracks 1&2: 1st Place on Zero-shot AD and 4th Place on Few-shot AD](https://mlsanity.com/paper/2305.17382/) (2023)
- [Mixed supervision for surface-defect detection: from weakly to fully supervised learning](https://mlsanity.com/paper/2104.06064/) (2021)
- [Deep Learning for Unsupervised Anomaly Localization in Industrial Images: A Survey](https://mlsanity.com/paper/2207.10298/) (2022)
- [Segment Any Anomaly without Training via Hybrid Prompt Regularization](https://mlsanity.com/paper/2305.10724/) (2023)
- [CLIP-AD: A Language-Guided Staged Dual-Path Model for Zero-shot Anomaly Detection](https://mlsanity.com/paper/2311.00453/) (2023)
- [ClipSAM: CLIP and SAM Collaboration for Zero-Shot Anomaly Segmentation](https://mlsanity.com/paper/2401.12665/) (2024)
- [Industrial Machine Tool Component Surface Defect Dataset](https://mlsanity.com/paper/2103.13003/) (2021)

## Built on by (most cited first, 30 of 50)

- [AA-CLIP: Enhancing Zero-shot Anomaly Detection via Anomaly-Aware CLIP](https://mlsanity.com/paper/2503.06661/) (2025)
- [Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey](https://mlsanity.com/paper/2407.21794/) (2024)
- [Bayesian Prompt Flow Learning for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2503.10080/) (2025)
- [AdaptCLIP: Adapting CLIP for Universal Visual Anomaly Detection](https://mlsanity.com/paper/2505.09926/) (2025)
- [MANTA: A Large-Scale Multi-View and Visual-Text Anomaly Detection Dataset for Tiny Objects](https://mlsanity.com/paper/2412.04867/) (2024)
- [Prompt-based Adaptation in Large-scale Vision Models: A Survey](https://mlsanity.com/paper/2510.13219/) (2025)
- [VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer](https://mlsanity.com/paper/2603.07952/) (2026)
- [DictAS: A Framework for Class-Generalizable Few-Shot Anomaly Segmentation via Dictionary Lookup](https://mlsanity.com/paper/2508.13560/) (2025)
- [MRAD: Zero-Shot Anomaly Detection with Memory-Driven Retrieval](https://mlsanity.com/paper/2602.00522/) (2026)
- [RAD: A Dataset and Benchmark for Real-Life Anomaly Detection with Robotic Observations](https://mlsanity.com/paper/2410.00713/) (2024)
- [A Survey on Foundation-Model-Based Industrial Defect Detection](https://mlsanity.com/paper/2502.19106/) (2025)
- [UltraAD: Fine-Grained Ultrasound Anomaly Classification via Few-Shot CLIP Adaptation](https://mlsanity.com/paper/2506.19694/) (2025)
- [Continual-MEGA: A Large-scale Benchmark for Generalizable Continual Anomaly Detection](https://mlsanity.com/paper/2506.00956/) (2025)
- [One Language-Free Foundation Model Is Enough for Universal Vision Anomaly Detection](https://mlsanity.com/paper/2601.05552/) (2026)
- [GS-CLIP: Zero-shot 3D Anomaly Detection by Geometry-Aware Prompt and Synergistic View Representation Learning](https://mlsanity.com/paper/2602.19206/) (2026)
- [ONER: Online Experience Replay for Incremental Anomaly Detection](https://mlsanity.com/paper/2412.03907/) (2024)
- [Crane: Context-Guided Prompt Learning and Attention Refinement for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2504.11055/) (2025)
- [GenCLIP: Generalizing CLIP Prompts for Zero-shot Anomaly Detection](https://mlsanity.com/paper/2504.14919/) (2025)
- [MuSc-V2: Zero-Shot Multimodal Industrial Anomaly Classification and Segmentation with Mutual Scoring of Unlabeled Samples](https://mlsanity.com/paper/2511.10047/) (2025)
- [PA-CLIP: Enhancing Zero-Shot Anomaly Detection through Pseudo-Anomaly Awareness](https://mlsanity.com/paper/2503.01292/) (2025)
- [Marine Saliency Segmenter: Object-Focused Conditional Diffusion with Region-Level Semantic Knowledge Distillation](https://mlsanity.com/paper/2504.02391/) (2025)
- [Quality-Aware Language-Conditioned Local Auto-Regressive Anomaly Synthesis and Detection](https://mlsanity.com/paper/2508.03539/) (2025)
- [Real-IAD Variety: Pushing Industrial Anomaly Detection Dataset to a Modern Era](https://mlsanity.com/paper/2511.00540/) (2025)
- [PromptMoE: Generalizable Zero-Shot Anomaly Detection via Visually-Guided Prompt Mixtures](https://mlsanity.com/paper/2511.18116/) (2025)
- [VitaTouch: Property-Aware Vision-Tactile-Language Model for Robotic Quality Inspection in Manufacturing](https://mlsanity.com/paper/2604.03322/) (2026)
- [IQE-CLIP: Instance-aware Query Embedding for Zero-/Few-shot Anomaly Detection in Medical Domain](https://mlsanity.com/paper/2506.10730/) (2025)
- [Toward Long-Tailed Online Anomaly Detection through Class-Agnostic Concepts](https://mlsanity.com/paper/2507.16946/) (2025)
- [CoPS: Conditional Prompt Synthesis for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2508.03447/) (2025)
- [A Contrastive Learning-Guided Confident Meta-learning for Zero Shot Anomaly Detection](https://mlsanity.com/paper/2508.17827/) (2025)
- [PointAD+: Learning Hierarchical Representations for Zero-shot 3D Anomaly Detection](https://mlsanity.com/paper/2509.03277/) (2025)

## Abstract (arXiv)

Recently, large-scale vision-language models such as CLIP have demonstrated immense potential in zero-shot anomaly segmentation (ZSAS) task, utilizing a unified model to directly detect anomalies on any unseen product with painstakingly crafted text prompts. However, existing methods often assume that the product category to be inspected is known, thus setting product-specific text prompts, which is difficult to achieve in the data privacy scenarios. Moreover, even the same type of product exhibits significant differences due to specific components and variations in the production process, posing significant challenges to the design of text prompts. In this end, we propose a visual context prompting model (VCP-CLIP) for ZSAS task based on CLIP. The insight behind VCP-CLIP is to employ visual context prompting to activate CLIP's anomalous semantic perception ability. In specific, we first design a Pre-VCP module to embed global visual information into the text prompt, thus eliminating the necessity for product-specific prompts. Then, we propose a novel Post-VCP module, that adjusts the text embeddings utilizing the fine-grained features of the images. In extensive experiments conducted on 10 real-world industrial anomaly segmentation datasets, VCP-CLIP achieved state-of-the-art performance in ZSAS task. The code is available at https://github.com/xiaozhen228/VCP-CLIP.