# PromptCAL: Contrastive Affinity Learning via Auxiliary Prompts for Generalized Novel Category Discovery

arXiv: https://arxiv.org/abs/2212.05590 · HTML: https://mlsanity.com/paper/2212.05590/

## Key facts

- Cited by 87 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 73 references that are in the map.
- First posted on arXiv 11 Dec 2022.

## Builds on (most cited first, 30 of 40)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [Deeply-Supervised Nets](https://mlsanity.com/paper/1409.5185/) (2014)
- [S4L: Self-Supervised Semi-Supervised Learning](https://mlsanity.com/paper/1905.03670/) (2019)
- [With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2104.14548/) (2021)
- [Recent Advances in Open Set Recognition: A Survey](https://mlsanity.com/paper/1811.08581/) (2018)
- [Label Propagation for Deep Semi-supervised Learning](https://mlsanity.com/paper/1904.04717/) (2019)
- [PTR: Prompt Tuning with Rules for Text Classification](https://mlsanity.com/paper/2105.11259/) (2021)
- [Learning to Discover Novel Visual Categories via Deep Transfer Clustering](https://mlsanity.com/paper/1908.09884/) (2019)
- [Generalized Category Discovery](https://mlsanity.com/paper/2201.02609/) (2022)
- [Local Aggregation for Unsupervised Learning of Visual Embeddings](https://mlsanity.com/paper/1903.12355/) (2019)
- [CoMatch: Semi-supervised Learning with Contrastive Graph Regularization](https://mlsanity.com/paper/2011.11183/) (2020)
- [Open-World Semi-Supervised Learning](https://mlsanity.com/paper/2102.03526/) (2021)
- [A Unified Objective for Novel Class Discovery](https://mlsanity.com/paper/2108.08536/) (2021)
- [Automatically Discovering and Learning New Visual Categories with Ranking Statistics](https://mlsanity.com/paper/2002.05714/) (2020)
- [Neighborhood Contrastive Learning for Novel Class Discovery](https://mlsanity.com/paper/2106.10731/) (2021)
- [Boosting Contrastive Self-Supervised Learning with False Negative Cancellation](https://mlsanity.com/paper/2011.11765/) (2020)

## Built on by (most cited first, 30 of 87)

- [What does CLIP know about a red circle? Visual prompt engineering for VLMs](https://mlsanity.com/paper/2304.06712/) (2023)
- [Learning Semi-supervised Gaussian Mixture Models for Generalized Category Discovery](https://mlsanity.com/paper/2305.06144/) (2023)
- [No Representation Rules Them All in Category Discovery](https://mlsanity.com/paper/2311.17055/) (2023)
- [Learn to Categorize or Categorize to Learn? Self-Coding for Generalized Category Discovery](https://mlsanity.com/paper/2310.19776/) (2023)
- [Contrastive Mean-Shift Learning for Generalized Category Discovery](https://mlsanity.com/paper/2404.09451/) (2024)
- [SelEx: Self-Expertise in Fine-Grained Generalized Category Discovery](https://mlsanity.com/paper/2408.14371/) (2024)
- [Active Generalized Category Discovery](https://mlsanity.com/paper/2403.04272/) (2024)
- [Textual Knowledge Matters: Cross-Modality Co-Teaching for Generalized Visual Class Discovery](https://mlsanity.com/paper/2403.07369/) (2024)
- [GET: Unlocking the Multi-modal Potential of CLIP for Generalized Category Discovery](https://mlsanity.com/paper/2403.09974/) (2024)
- [Solving the Catastrophic Forgetting Problem in Generalized Category Discovery](https://mlsanity.com/paper/2501.05272/) (2025)
- [HiLo: A Learning Framework for Generalized Category Discovery Robust to Domain Shifts](https://mlsanity.com/paper/2408.04591/) (2024)
- [A Graph-Theoretic Framework for Understanding Open-World Semi-Supervised Learning](https://mlsanity.com/paper/2311.03524/) (2023)
- [Hyperbolic Category Discovery](https://mlsanity.com/paper/2504.06120/) (2025)
- [Happy: A Debiased Learning Framework for Continual Generalized Category Discovery](https://mlsanity.com/paper/2410.06535/) (2024)
- [DebGCD: Debiased Learning with Distribution Guidance for Generalized Category Discovery](https://mlsanity.com/paper/2504.04804/) (2025)
- [Effective Prompt Pool Learning for Continual Category Discovery](https://mlsanity.com/paper/2407.19001/) (2024)
- [Adaptive Part Learning for Fine-Grained Generalized Category Discovery: A Plug-and-Play Enhancement](https://mlsanity.com/paper/2507.06928/) (2025)
- [Dissecting Generalized Category Discovery: Multiplex Consensus under Self-Deconstruction](https://mlsanity.com/paper/2508.10731/) (2025)
- [A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models](https://mlsanity.com/paper/2307.12980/) (2023)
- [CDAD-Net: Bridging Domain Gaps in Generalized Category Discovery](https://mlsanity.com/paper/2404.05366/) (2024)
- [Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling](https://mlsanity.com/paper/2403.01053/) (2024)
- [Dual-level Adaptive Self-Labeling for Novel Class Discovery in Point Cloud Segmentation](https://mlsanity.com/paper/2407.12489/) (2024)
- [Flipped Classroom: Aligning Teacher Attention with Student in Generalized Category Discovery](https://mlsanity.com/paper/2409.19659/) (2024)
- [Multimodal Generalized Category Discovery](https://mlsanity.com/paper/2409.11624/) (2024)
- [SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery](https://mlsanity.com/paper/2510.18740/) (2025)
- [Contextuality Helps Representation Learning for Generalized Category Discovery](https://mlsanity.com/paper/2407.19752/) (2024)
- [Interpretable Zero-Shot Learning with Locally-Aligned Vision-Language Model](https://mlsanity.com/paper/2506.23822/) (2025)
- [Consistent Supervised-Unsupervised Alignment for Generalized Category Discovery](https://mlsanity.com/paper/2507.04725/) (2025)
- [Category Adaptation Meets Projected Distillation in Generalized Continual Category Discovery](https://mlsanity.com/paper/2308.12112/) (2023)
- [When Domain Generalization meets Generalized Category Discovery: An Adaptive Task-Arithmetic Driven Approach](https://mlsanity.com/paper/2503.14897/) (2025)

## Abstract (arXiv)

Although existing semi-supervised learning models achieve remarkable success in learning with unannotated in-distribution data, they mostly fail to learn on unlabeled data sampled from novel semantic classes due to their closed-set assumption. In this work, we target a pragmatic but under-explored Generalized Novel Category Discovery (GNCD) setting. The GNCD setting aims to categorize unlabeled training data coming from known and novel classes by leveraging the information of partially labeled known classes. We propose a two-stage Contrastive Affinity Learning method with auxiliary visual Prompts, dubbed PromptCAL, to address this challenging problem. Our approach discovers reliable pairwise sample affinities to learn better semantic clustering of both known and novel classes for the class token and visual prompts. First, we propose a discriminative prompt regularization loss to reinforce semantic discriminativeness of prompt-adapted pre-trained vision transformer for refined affinity relationships.Besides, we propose contrastive affinity learning to calibrate semantic representations based on our iterative semi-supervised affinity graph generation method for semantically-enhanced supervision. Extensive experimental evaluation demonstrates that our PromptCAL method is more effective in discovering novel classes even with limited annotations and surpasses the current state-of-the-art on generic and fine-grained benchmarks (e.g., with nearly 11% gain on CUB-200, and 9% on ImageNet-100) on overall accuracy. Our code is available at https://github.com/sheng-eatamath/PromptCAL.