mlsanityData to 7 Oct 2026

Generalized Category Discovery under Domain Shifts: From Vision to Vision-Language Models

arXiv 2605.00906 ↗First posted 29 Apr 2026cs.CV cs.AI cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Generalized Category Discovery (GCD) aims to categorize unlabelled instances from both known and unknown classes by transferring knowledge from labelled data of known classes. Existing methods assume all data comes from a single domain, yet real-world unlabelled data often exhibits domain shifts alongside semantic shifts. We study GCD under domain shifts and propose three frameworks that adapt foundation models, ranging from self-supervised vision models to vision-language models. (i) HiLo disentangles domain and semantic features through multi-level feature extraction and mutual information minimization, combined with PatchMix augmentation and curriculum sampling. (ii) HLPrompt extends HiLo with semantic-aware spatial prompt tuning to suppress background and domain noise. (iii) VLPrompt leverages vision-language models via factorized textual prompts and cross-modal consistency regularization. The three methods share core design principles while operating on different foundation backbones, making them suitable for different deployment scenarios. Extensive experiments on synthetic corruptions and real-world multi-domain shifts demonstrate consistent improvements over strong baselines. Project page: https://visual-ai.github.io/hilo/

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 84 of its 113 references that are in the map. First posted on arXiv 29 Apr 2026.

Builds on

84 of 113 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 83

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  8. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  9. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  10. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  11. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  12. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
38 more
  1. 2022Visual Prompt Tuningcited by 1,674
  2. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  3. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  4. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,327
  5. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,323
  6. 2017Adversarial Discriminative Domain Adaptationcited by 1,309
  7. 2025DINOv3cited by 1,297
  8. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  9. 2014How transferable are features in deep neural networks?cited by 1,216
  10. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  11. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  12. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  13. 2019Introduction to Online Convex Optimizationcited by 793
  14. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  15. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  16. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 748
  17. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  18. 2013Domain Generalization via Invariant Feature Representationcited by 622
  19. 2016Deep Transfer Learning with Joint Adaptation Networkscited by 609
  20. 2014Deep Domain Confusion: Maximizing for Domain Invariancecited by 597
  21. 2021Domain Generalization with MixStylecited by 563
  22. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  23. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 431
  24. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  25. 2021Open-Set Recognition: a Good Closed-Set Classifier is All You Need?cited by 351
  26. 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 311
  27. 2022Masked Siamese Networks for Label-Efficient Learningcited by 301
  28. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 288
  29. 2019On The Power of Curriculum Learning in Training Deep Networkscited by 285
  30. 2021Gradient Matching for Domain Generalizationcited by 258
  31. 2019Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptationcited by 235
  32. 2020Domain Generalization using Causal Matchingcited by 224
  33. 2019Learning to Discover Novel Visual Categories via Deep Transfer Clusteringcited by 223
  34. 2022Generalized Category Discoverycited by 209
  35. 2021Open-World Semi-Supervised Learningcited by 165
  36. 2021A Unified Objective for Novel Class Discoverycited by 161
  37. 2020Automatically Discovering and Learning New Visual Categories with Ranking Statisticscited by 141
  38. 2020FixBi: Bridging Domain Spaces for Unsupervised Domain Adaptationcited by 124

50 most cited of 83 All 83 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Test-Time Generalized Category Discovery
  2. 2024GraphVL: Graph-Enhanced Semantic Modeling via Vision-Language Models for Generalized Class Discovery
  3. 2025HIDISC: A Hyperbolic Framework for Domain Generalization with Generalized Category Discovery
  4. 2025Explaining Domain Shifts in Language: Concept erasing for Interpretable Image Classification
  5. 2025Latent Domain Prompt Learning for Vision-Language Models
  6. 2024Textual Knowledge Matters: Cross-Modality Co-Teaching for Generalized Visual Class Discovery
  7. 2025GLEAN: Active Generalized Category Discovery with Diverse LLM Feedback
  8. 2024Unknown Prompt, the only Lacuna: Unveiling CLIP's Potential for Open Domain Generalization