# VQ-Flow: Taming Normalizing Flows for Multi-Class Anomaly Detection via Hierarchical Vector Quantization

arXiv: https://arxiv.org/abs/2409.00942 · HTML: https://mlsanity.com/paper/2409.00942/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 44 references that are in the map.
- First posted on arXiv 2 Sep 2024.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023)
- [PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization](https://mlsanity.com/paper/2011.08785/) (2020)
- [DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detection](https://mlsanity.com/paper/2108.07610/) (2021)
- [SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation](https://mlsanity.com/paper/2207.14315/) (2022)
- [Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings](https://mlsanity.com/paper/1911.02357/) (2019)
- [CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows](https://mlsanity.com/paper/2107.12571/) (2021)
- [Multiresolution Knowledge Distillation for Anomaly Detection](https://mlsanity.com/paper/2011.11108/) (2020)
- [A Unified Model for Multi-class Anomaly Detection](https://mlsanity.com/paper/2206.03687/) (2022)
- [SimpleNet: A Simple Network for Image Anomaly Detection and Localization](https://mlsanity.com/paper/2303.15140/) (2023)
- [Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision](https://mlsanity.com/paper/2007.04687/) (2020)
- [PANDA: Adapting Pretrained Features for Anomaly Detection and Segmentation](https://mlsanity.com/paper/2010.05903/) (2020)
- [Explainable Deep One-Class Classification](https://mlsanity.com/paper/2007.01760/) (2020)
- [Hierarchical Vector Quantized Transformer for Multi-class Unsupervised Anomaly Detection](https://mlsanity.com/paper/2310.14228/) (2023)
- [Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detection](https://mlsanity.com/paper/2312.07495/) (2023)
- [Unified Multivariate Gaussian Mixture for Efficient Neural Image Compression](https://mlsanity.com/paper/2203.10897/) (2022)
- [Self-Supervised Masking for Unsupervised Anomaly Detection and Localization](https://mlsanity.com/paper/2205.06568/) (2022)
- [DiAD: A Diffusion-based Framework for Multi-class Anomaly Detection](https://mlsanity.com/paper/2312.06607/) (2023)
- [Stacked Quantizers for Compositional Vector Compression](https://mlsanity.com/paper/1411.2173/) (2014)
- [BatchNorm-based Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2311.15367/) (2023)

## Built on by (most cited first, 5 of 5)

- [A Survey on Foundation-Model-Based Industrial Defect Detection](https://mlsanity.com/paper/2502.19106/) (2025)
- [Multi-class Image Anomaly Detection for Practical Applications: Requirements and Robust Solutions](https://mlsanity.com/paper/2508.02477/) (2025)
- [3D Human-Human Interaction Anomaly Detection](https://mlsanity.com/paper/2512.13560/) (2025)
- [Modulate-and-Map: Crossmodal Feature Mapping with Cross-View Modulation for 3D Anomaly Detection](https://mlsanity.com/paper/2604.02328/) (2026)
- [DeCoFlow: Structural Decomposition of Normalizing Flows for Continual Anomaly Detection](https://mlsanity.com/paper/2606.26687/) (2026)

## Abstract (arXiv)

Normalizing flows, a category of probabilistic models famed for their capabilities in modeling complex data distributions, have exhibited remarkable efficacy in unsupervised anomaly detection. This paper explores the potential of normalizing flows in multi-class anomaly detection, wherein the normal data is compounded with multiple classes without providing class labels. Through the integration of vector quantization (VQ), we empower the flow models to distinguish different concepts of multi-class normal data in an unsupervised manner, resulting in a novel flow-based unified method, named VQ-Flow. Specifically, our VQ-Flow leverages hierarchical vector quantization to estimate two relative codebooks: a Conceptual Prototype Codebook (CPC) for concept distinction and its concomitant Concept-Specific Pattern Codebook (CSPC) to capture concept-specific normal patterns. The flow models in VQ-Flow are conditioned on the concept-specific patterns captured in CSPC, capable of modeling specific normal patterns associated with different concepts. Moreover, CPC further enables our VQ-Flow for concept-aware distribution modeling, faithfully mimicking the intricate multi-class normal distribution through a mixed Gaussian distribution reparametrized on the conceptual prototypes. Through the introduction of vector quantization, the proposed VQ-Flow advances the state-of-the-art in multi-class anomaly detection within a unified training scheme, yielding the Det./Loc. AUROC of 99.5%/98.3% on MVTec AD. The codebase is publicly available at https://github.com/cool-xuan/vqflow.