mlsanityData to 7 Oct 2026

VLMDiff: Leveraging Vision-Language Models for Multi-Class Anomaly Detection with Diffusion

arXiv 2511.08173 ↗First posted 11 Nov 2025cs.CVcited by 3

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

also in 1 more
Abstract (arXiv)

Detecting visual anomalies in diverse, multi-class real-world images is a significant challenge. We introduce ours, a novel unsupervised multi-class visual anomaly detection framework. It integrates a Latent Diffusion Model (LDM) with a Vision-Language Model (VLM) for enhanced anomaly localization and detection. Specifically, a pre-trained VLM with a simple prompt extracts detailed image descriptions, serving as additional conditioning for LDM training. Current diffusion-based methods rely on synthetic noise generation, limiting their generalization and requiring per-class model training, which hinders scalability. ours, however, leverages VLMs to obtain normal captions without manual annotations or additional training. These descriptions condition the diffusion model, learning a robust normal image feature representation for multi-class anomaly detection. Our method achieves competitive performance, improving the pixel-level Per-Region-Overlap (PRO) metric by up to 25 points on the Real-IAD dataset and 8 points on the COCO-AD dataset, outperforming state-of-the-art diffusion-based approaches. Code is available at https://github.com/giddyyupp/VLMDiff.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 53 of its 69 references that are in the map. First posted on arXiv 11 Nov 2025.

Builds on

53 of 69 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Models11 Feb

Other references · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  11. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  12. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
38 more
  1. 2013Describing Textures in the Wildcited by 2,110
  2. 2021Towards Total Recall in Industrial Anomaly Detectioncited by 772
  3. 2020PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localizationcited by 591
  4. 2023Segment Everything Everywhere All at Oncecited by 546
  5. 2021DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detectioncited by 510
  6. 2022Anomaly Detection via Reverse Distillation from One-Class Embeddingcited by 419
  7. 2018GANomaly: Semi-Supervised Anomaly Detection via Adversarial Trainingcited by 409
  8. 2022SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentationcited by 397
  9. 2019Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddingscited by 386
  10. 2021CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flowscited by 320
  11. 2023WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentationcited by 311
  12. 2020Sub-Image Anomaly Detection with Deep Pyramid Correspondencescited by 297
  13. 2023AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detectioncited by 270
  14. 2022A Unified Model for Multi-class Anomaly Detectioncited by 267
  15. 2018Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoderscited by 263
  16. 2023SimpleNet: A Simple Network for Image Anomaly Detection and Localizationcited by 256
  17. 2022Diffusion Models for Medical Anomaly Detectioncited by 231
  18. 2023A Simple Framework for Open-Vocabulary Segmentation and Detectioncited by 197
  19. 2020Deep Learning for Anomaly Detection: A Reviewcited by 159
  20. 2022DeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detectioncited by 153
  21. 2024Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detectioncited by 138
  22. 2022CFA: Coupled-hypersphere-based Feature Adaptation for Target-Oriented Anomaly Localizationcited by 135
  23. 2024AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detectioncited by 126
  24. 2024RealNet: A Feature Selection Network with Realistic Synthetic Anomaly for Anomaly Detectioncited by 116
  25. 2024MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detectioncited by 113
  26. 2022Anomaly Detection in Autonomous Driving: A Surveycited by 87
  27. 2024Harnessing Large Language Models for Training-free Video Anomaly Detectioncited by 85
  28. 2023PyramidFlow: High-Resolution Defect Contrastive Localization using Pyramid Normalizing Flowcited by 74
  29. 2023On Diffusion Modeling for Anomaly Detectioncited by 71
  30. 2024FiLo: Zero-Shot Anomaly Detection by Fine-Grained Description and High-Quality Localizationcited by 51
  31. 2023Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detectioncited by 47
  32. 2023DiffusionAD: Norm-guided One-step Denoising Diffusion for Anomaly Detectioncited by 46
  33. 2023TransFusion -- A Transparency-Based Diffusion Model for Anomaly Detectioncited by 43
  34. 2024GLAD: Towards Better Reconstruction with Global and Local Adaptive Diffusion Models for Unsupervised Anomaly Detectioncited by 42
  35. 2024Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Surveycited by 36
  36. 2022Exposing Outlier Exposure: What Can Be Learned From Few, One, and Zero Outlier Imagescited by 35
  37. 2024Large Language Models for Anomaly and Out-of-Distribution Detection: A Surveycited by 24
  38. 2022A Survey on Unsupervised Anomaly Detection Algorithms for Industrial Imagescited by 21

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Training Free Zero-Shot Visual Anomaly Localization via Diffusion Inversioncited by 1
  2. 2026SwinAD: Multi-stage feature reconstruction for unsupervised industrial anomaly detectioncited by 0
  3. 2026Multi-View Reflective Surface Inspection via Semantic-Saliency Cross-Verificationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Video Anomaly Detection in 10 Years: A Survey and Outlook
  2. 2025From Benchmarks to Reality: Advancing Visual Anomaly Detection by the VAND 3.0 Challenge
  3. 2024Quo Vadis, Anomaly Detection? LLMs and VLMs in the Spotlight
  4. 2024Do LLMs Understand Visual Anomalies? Uncovering LLM's Capabilities in Zero-shot Anomaly Detection
  5. 2024Vision-Language Models Assisted Unsupervised Video Anomaly Detection
  6. 2024VL4AD: Vision-Language Models Improve Pixel-wise Anomaly Detection
  7. 2025Explainable Deep Convolutional Multi-Type Anomaly Detection
  8. 2026AnomalyVFM -- Transforming Vision Foundation Models into Zero-Shot Anomaly Detectors