mlsanityData to 14 Oct 2026

Segment Any Anomaly without Training via Hybrid Prompt Regularization

arXiv 2305.10724 ↗First posted 18 May 2023cs.CV cs.AIcited by 60

Not on the map. Why?

Abstract (arXiv)

We present a novel framework, i.e., Segment Any Anomaly + (SAA+), for zero-shot anomaly segmentation with hybrid prompt regularization to improve the adaptability of modern foundation models. Existing anomaly segmentation models typically rely on domain-specific fine-tuning, limiting their generalization across countless anomaly patterns. In this work, inspired by the great zero-shot generalization ability of foundation models like Segment Anything, we first explore their assembly to leverage diverse multi-modal prior knowledge for anomaly localization. For non-parameter foundation model adaptation to anomaly segmentation, we further introduce hybrid prompts derived from domain expert knowledge and target image context as regularization. Our proposed SAA+ model achieves state-of-the-art performance on several anomaly segmentation benchmarks, including VisA, MVTec-AD, MTD, and KSDD2, in the zero-shot setting. We will release the code at hrefhttps://github.com/caoyunkang/Segment-Any-Anomalyhttps://github.com/caoyunkang/Segment-Any-Anomaly.

Key facts. Cited by 60 papers in the map (data through 14 Oct 2026). Builds on 57 of its 107 references that are in the map. First posted on arXiv 18 May 2023.

Builds on

57 of 107 references.

References · 57

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023Segment Anythingcited by 8,234
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  6. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  7. 2016Wide Residual Networkscited by 3,070
  8. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  9. 2022Visual Prompt Tuningcited by 1,681
  10. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  11. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  12. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
38 more
  1. 2021Grounded Language-Image Pre-trainingcited by 1,156
  2. 2021Towards Total Recall in Industrial Anomaly Detectioncited by 776
  3. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 616
  4. 2021Extract Free Dense Labels from CLIPcited by 583
  5. 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 532
  6. 2021DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detectioncited by 514
  7. 2021Image Segmentation Using Text and Image Promptscited by 512
  8. 2022Anomaly Detection via Reverse Distillation from One-Class Embeddingcited by 421
  9. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 421
  10. 2022SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentationcited by 400
  11. 2019Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddingscited by 386
  12. 2023WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentationcited by 315
  13. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 310
  14. 2020Multiresolution Knowledge Distillation for Anomaly Detectioncited by 285
  15. 2022Exploring Visual Prompts for Adapting Large-Scale Modelscited by 274
  16. 2023What does CLIP know about a red circle? Visual prompt engineering for VLMscited by 218
  17. 2020Patch SVDD: Patch-level SVDD for Anomaly Detection and Segmentationcited by 212
  18. 2023Teaching CLIP to Count to Tencited by 198
  19. 2019Salient Object Detection in the Deep Learning Era: An In-Depth Surveycited by 164
  20. 2021Align and Prompt: Video-and-Language Pre-training with Entity Promptscited by 149
  21. 2022Unified Vision and Language Prompt Learningcited by 148
  22. 2020Autoencoders for Unsupervised Anomaly Segmentation in Brain MR Images: A Comparative Studycited by 147
  23. 2021Student-Teacher Feature Pyramid Matching for Anomaly Detectioncited by 146
  24. 2023Multimodal Industrial Anomaly Detection via Hybrid Fusioncited by 139
  25. 2020Learning and Evaluating Representations for Deep One-class Classificationcited by 133
  26. 2022Registration based Few-Shot Anomaly Detectioncited by 133
  27. 2021Mixed supervision for surface-defect detection: from weakly to fully supervised learningcited by 101
  28. 2021Divide-and-Assemble: Learning Block-wise Memory for Unsupervised Anomaly Detectioncited by 83
  29. 2022Omni-frequency Channel-selection Representations for Unsupervised Anomaly Detectioncited by 80
  30. 2023Complementary Pseudo Multimodal Feature for Point Cloud Anomaly Detectioncited by 75
  31. 2022Explicit Boundary Guided Semi-Push-Pull Contrastive Learning for Supervised Anomaly Detectioncited by 72
  32. 2023IM-IAD: Industrial Image Anomaly Detection Benchmark in Manufacturingcited by 64
  33. 2023Collaborative Discrepancy Optimization for Reliable Image Anomaly Localizationcited by 57
  34. 2020Encoding Structure-Texture Relation with P-Net for Anomaly Detection in Retinal Imagescited by 52
  35. 2022Multitask Vision-Language Prompt Tuningcited by 51
  36. 2022Self-Supervised Masking for Unsupervised Anomaly Detection and Localizationcited by 43
  37. 2021Reliable Propagation-Correction Modulation for Video Object Segmentationcited by 36
  38. 2020MOCCA: Multi-Layer One-Class ClassificAtion for Anomaly Detectioncited by 31

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 17; 2024: 20; 2025: 15; 2026: 8.

60 papers.

Sort:

By area:

  1. 2024AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detectioncited by 127
  2. 2023AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Modelcited by 111
  3. 2024PromptAD: Learning Prompts with only Normal Samples for Few-Shot Anomaly Detectioncited by 98
  4. 2023Real3D-AD: A Dataset of Point Cloud Anomaly Detectioncited by 87
  5. 2023Review of Large Vision Models and Visual Prompt Engineeringcited by 77
  6. 2025AA-CLIP: Enhancing Zero-shot Anomaly Detection via Anomaly-Aware CLIPcited by 73
  7. 2024A Survey on Visual Anomaly Detection: Challenge, Approach, and Prospectcited by 69
  8. 2023Towards Scalable 3D Anomaly Detection and Localization: A Benchmark via 3D Anomaly Synthesis and A Self-Supervised Learning Networkcited by 67
  9. 2023CLIP-AD: A Language-Guided Staged Dual-Path Model for Zero-shot Anomaly Detectioncited by 58
  10. 2024MuSc: Zero-Shot Industrial Anomaly Classification and Segmentation with Mutual Scoring of the Unlabeled Imagescited by 52
  11. 2024FiLo: Zero-Shot Anomaly Detection by Fine-Grained Description and High-Quality Localizationcited by 51
  12. 2024VCP-CLIP: A visual context prompting model for zero-shot anomaly segmentationcited by 50
38 more
  1. 2024ClipSAM: CLIP and SAM Collaboration for Zero-Shot Anomaly Segmentationcited by 49
  2. 2023Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detectioncited by 47
  3. 2023Towards Generic Anomaly Detection and Understanding: Large-scale Visual-linguistic Model (GPT-4V) Takes the Leadcited by 40
  4. 2023DiAD: A Diffusion-based Framework for Multi-class Anomaly Detectioncited by 33
  5. 2023GPT-4V-AD: Exploring Grounding Potential of VQA-oriented GPT-4V for Zero-shot Anomaly Detectioncited by 23
  6. 2023Target before Shooting: Accurate Anomaly Detection and Localization under One Millisecond via Cascade Patch Retrievalcited by 22
  7. 2024AnomalyXFusion: Multi-modal Anomaly Synthesis with Diffusioncited by 21
  8. 2024Spiking-PhysFormer: Camera-Based Remote Photoplethysmography with Parallel Spike-driven Transformercited by 19
  9. 2025Learning to Detect Multi-class Anomalies with Just One Normal Image Promptcited by 18
  10. 2025OmniAD: Detect and Understand Industrial Anomaly via Multimodal Reasoningcited by 18
  11. 2023RepViT-SAM: Towards Real-Time Segmenting Anythingcited by 17
  12. 2025Boosting Global-Local Feature Matching via Anomaly Synthesis for Multi-Class Point Cloud Anomaly Detectioncited by 16
  13. 2023UGainS: Uncertainty Guided Anomaly Instance Segmentationcited by 15
  14. 2024Towards Zero-shot Point Cloud Anomaly Detection: A Multi-View Projection Frameworkcited by 15
  15. 2024Long-Tailed Anomaly Detection with Learnable Class Namescited by 14
  16. 2023Optimizing PatchCore for Few/many-shot Anomaly Detectioncited by 10
  17. 2024RAD: A Comprehensive Dataset for Benchmarking the Robustness of Image Anomaly Detectioncited by 8
  18. 2025UrbanSAM: Learning Invariance-Inspired Adapters for Segment Anything Models in Urban Constructioncited by 8
  19. 2024FADE: Few-shot/zero-shot Anomaly Detection Engine using Large Vision-Language Modelcited by 7
  20. 2024Promptable Anomaly Segmentation with SAM Through Self-Perception Tuningcited by 7
  21. 2025Center-aware Residual Anomaly Synthesis for Multi-class Industrial Anomaly Detectioncited by 7
  22. 2024Blind Localization and Clustering of Anomalies in Texturescited by 5
  23. 2025A Survey on Foundation-Model-Based Industrial Defect Detectioncited by 5
  24. 2023Industrial Segment Anything -- a Case Study in Aircraft Manufacturing, Intralogistics, Maintenance, Repair, and Overhaulcited by 4
  25. 2023Large Language Models and Foundation Models in Smart Agriculture: Basics, Opportunities, and Challengescited by 4
  26. 2024Global-Regularized Neighborhood Regression for Efficient Zero-Shot Texture Anomaly Detectioncited by 4
  27. 2025VADER: Towards Causal Video Anomaly Understanding with Relation-Aware Large Language Modelscited by 4
  28. 2024UMAD: Unsupervised Mask-Level Anomaly Detection for Autonomous Drivingcited by 3
  29. 2024Exploring Aleatoric Uncertainty in Object Detection via Vision Foundation Modelscited by 3
  30. 2025GenCLIP: Generalizing CLIP Prompts for Zero-shot Anomaly Detectioncited by 3
  31. 2025Prompt Engineering in Segment Anything Model: Methodologies, Applications, and Emerging Challengescited by 3
  32. 20232nd Place Winning Solution for the CVPR2023 Visual Anomaly and Novelty Detection Challenge: Multimodal Prompting for Data-centric Anomaly Detectioncited by 2
  33. 2023Zero-Shot Edge Detection with SCESAME: Spectral Clustering-based Ensemble for Segment Anything Model Estimationcited by 2
  34. 2026VTFusion: A Vision-Text Multimodal Fusion Network for Few-Shot Anomaly Detectioncited by 2
  35. 2026Back to Point: Exploring Point-Language Models for Zero-Shot 3D Anomaly Detectioncited by 2
  36. 2025Leveraging Learning Bias for Noisy Anomaly Detectioncited by 1
  37. 2025Towards Accurate UAV Image Perception: Guiding Vision-Language Models with Stronger Task Promptscited by 1
  38. 2026Training Free Zero-Shot Visual Anomaly Localization via Diffusion Inversioncited by 1

50 most cited of 60 All 60 (JSON)