# Dual Anchors, Do It Better: Hierarchical Group Merging for Zero-Shot Anomaly Detection

arXiv: https://arxiv.org/abs/2608.11933 · HTML: https://mlsanity.com/paper/2608.11933/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 31 references that are in the map.
- First posted on arXiv 12 Aug 2026.

## Builds on (most cited first, 22 of 22)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [GroupViT: Semantic Segmentation Emerges from Text Supervision](https://mlsanity.com/paper/2202.11094/) (2022)
- [SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation](https://mlsanity.com/paper/2207.14315/) (2022)
- [WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation](https://mlsanity.com/paper/2303.14814/) (2023)
- [AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection](https://mlsanity.com/paper/2310.18961/) (2023)
- [VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization](https://mlsanity.com/paper/2104.10036/) (2021)
- [Skin Lesion Analysis toward Melanoma Detection: A Challenge at the International Symposium on Biomedical Imaging (ISBI) 2016, hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1605.01397/) (2016)
- [APRIL-GAN: A Zero-/Few-Shot Anomaly Classification and Segmentation Method for CVPR 2023 VAND Workshop Challenge Tracks 1&2: 1st Place on Zero-shot AD and 4th Place on Few-shot AD](https://mlsanity.com/paper/2305.17382/) (2023)
- [TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?](https://mlsanity.com/paper/2106.11297/) (2021)
- [AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2407.15795/) (2024)
- [Mixed supervision for surface-defect detection: from weakly to fully supervised learning](https://mlsanity.com/paper/2104.06064/) (2021)
- [AA-CLIP: Enhancing Zero-shot Anomaly Detection via Anomaly-Aware CLIP](https://mlsanity.com/paper/2503.06661/) (2025)
- [CLIP-AD: A Language-Guided Staged Dual-Path Model for Zero-shot Anomaly Detection](https://mlsanity.com/paper/2311.00453/) (2023)
- [VCP-CLIP: A visual context prompting model for zero-shot anomaly segmentation](https://mlsanity.com/paper/2407.12276/) (2024)
- [HGFormer: Hierarchical Grouping Transformer for Domain Generalized Semantic Segmentation](https://mlsanity.com/paper/2305.13031/) (2023)
- [Bayesian Prompt Flow Learning for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2503.10080/) (2025)
- [Bootstrap Fine-Grained Vision-Language Alignment for Unified Zero-Shot Anomaly Localization](https://mlsanity.com/paper/2308.15939/) (2023)

## Abstract (arXiv)

Zero-shot anomaly detection (ZSAD) aims to identify anomalies in unseen domains, a setting that is particularly critical for industrial and medical applications where domain shifts are prevalent. However, most CLIP-based ZSAD methods anchor semantics solely on the text modality, making performance highly sensitive to prompt design and leading to weak visual grounding. To mitigate these limitations, we propose a Dual-Anchor framework that complements conventional text anchors with hierarchical image anchors constructed via a top-down grouping mechanism. This mechanism progressively aggregates local-to-global image features to form normal and abnormal group tokens, which serve as image anchors and act as gating signals in a Group-Gated Token Refiner to enhance the global representation. The refined image anchors are then fused with text prompts to construct dynamic state prompts. By jointly reinforcing visual and textual semantics, our framework stabilizes image-text alignment, reduces prompt dependency, and achieves strong generalization across 8 industrial and 6 medical benchmarks.