mlsanityData to 14 Oct 2026

A Sentence Speaks a Thousand Images: Domain Generalization through Distilling CLIP with Language Guidance

arXiv 2309.12530 ↗First posted 21 Sep 2023cs.CVcited by 40

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Domain generalization studies the problem of training a model with samples from several domains (or distributions) and then testing the model with samples from a new, unseen domain. In this paper, we propose a novel approach for domain generalization that leverages recent advances in large vision-language models, specifically a CLIP teacher model, to train a smaller model that generalizes to unseen domains. The key technical contribution is a new type of regularization that requires the student's learned image representations to be close to the teacher's learned text representations obtained from encoding the corresponding text descriptions of images. We introduce two designs of the loss function, absolute and relative distance, which provide specific guidance on how the training process of the student model should be regularized. We evaluate our proposed method, dubbed RISE (Regularized Invariance with Semantic Embeddings), on various benchmark datasets and show that it outperforms several state-of-the-art domain generalization methods. To our knowledge, our work is the first to leverage knowledge distillation using a large vision-language model for domain generalization. By incorporating text-based information, RISE improves the generalization capability of machine learning models.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 68 of its 86 references that are in the map. First posted on arXiv 21 Sep 2023.

Builds on

68 of 86 references.

References · 68

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  4. 2020Knowledge Distillation: A Surveycited by 1,603
  5. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
  6. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,200
  7. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,188
  8. 2020In Search of Lost Domain Generalizationcited by 1,054
  9. 2017Deeper, Broader and Artier Domain Generalizationcited by 986
  10. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 906
  11. 2019Relational Knowledge Distillationcited by 889
  12. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
38 more
  1. 2018Recognition in Terra Incognitacited by 658
  2. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 627
  3. 2013Domain Generalization via Invariant Feature Representationcited by 623
  4. 2021Extract Free Dense Labels from CLIPcited by 583
  5. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 463
  6. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 436
  7. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 434
  8. 2020Self-Challenging Improves Cross-Domain Generalizationcited by 426
  9. 2019A Comprehensive Overhaul of Feature Distillationcited by 348
  10. 2017Unified Deep Supervised Domain Adaptation and Generalizationcited by 323
  11. 2019Correlation Congruence for Knowledge Distillationcited by 285
  12. 2018Generalizing Across Domains via Cross-Gradient Trainingcited by 268
  13. 2018Paraphrasing Complex Network: Network Compression via Factor Transfercited by 260
  14. 2019Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Datacited by 246
  15. 2020Deep Domain-Adversarial Image Generation for Domain Generalisationcited by 231
  16. 2022Vision-Language Pre-Training with Triple Contrastive Learningcited by 212
  17. 2022Unified Contrastive Learning in Image-Text-Label Spacecited by 194
  18. 2018DLOW: Domain Flow for Adaptation and Generalizationcited by 177
  19. 2021FSDR: Frequency Space Domain Randomization for Domain Generalizationcited by 164
  20. 2019Learning to Optimize Domain Specific Normalization for Domain Generalizationcited by 161
  21. 2021Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalizationcited by 144
  22. 2020Knowledge Distillation Meets Self-Supervisioncited by 142
  23. 2022Domain Generalization by Mutual-Information Regularization with Pre-trained Modelscited by 133
  24. 2021OoD-Bench: Quantifying and Understanding Two Dimensions of Out-of-Distribution Generalizationcited by 86
  25. 2020Heterogeneous Knowledge Distillation using Information Flow Modelingcited by 73
  26. 2022Domain Generalization via Shuffled Style Assembly for Face Anti-Spoofingcited by 73
  27. 2020ALP-KD: Attention-Based Layer Projection for Knowledge Distillationcited by 66
  28. 2017DarkRank: Accelerating Deep Metric Learning via Cross Sample Similarities Transfercited by 56
  29. 2021Domain Invariant Representation Learning with Domain Density Transformationscited by 53
  30. 2018Learning Student Networks via Feature Embeddingcited by 45
  31. 2022Out-of-distribution Generalization with Causal Invariant Transformationscited by 45
  32. 2019Adversarial Invariant Feature Learning with Accuracy Constraint for Domain Generalizationcited by 43
  33. 2019Learning Metrics from Teachers: Compact Networks for Image Embeddingcited by 39
  34. 2021Domain Prompt Learning for Efficiently Adapting CLIP to Unseen Domainscited by 39
  35. 2018Amalgamating Knowledge towards Comprehensive Classificationcited by 38
  36. 2022Learning Visual Representation from Modality-Shared Contrastive Language-Image Pre-trainingcited by 32
  37. 2016Select-Additive Learning: Improving Generalization in Multimodal Sentiment Analysiscited by 29
  38. 2021Embracing the Dark Knowledge: Domain Generalization Using Regularized Knowledge Distillationcited by 29

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 14; 2025: 18; 2026: 4.

40 papers.

Sort:

By area:

  1. 2023Leveraging Vision-Language Models for Improving Domain Generalization in Image Classificationcited by 41
  2. 2025Mamba as a Bridge: Where Vision Foundation Models Meet Vision Language Models for Domain-Generalized Semantic Segmentationcited by 24
  3. 2025Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Modelscited by 23
  4. 2024Textual Query-Driven Mask Transformer for Domain Generalized Segmentationcited by 21
  5. 2024CLIP-Guided Generative Networks for Transferable Targeted Adversarial Attackscited by 19
  6. 2024PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalizationcited by 15
  7. 2025CLIP-Powered Domain Generalization and Domain Adaptation: A Comprehensive Surveycited by 15
  8. 2024Learn to Preserve and Diversify: Parameter-Efficient Group with Orthogonal Regularization for Domain Generalizationcited by 14
  9. 2024QT-DoG: Quantization-aware Training for Domain Generalizationcited by 13
  10. 2023Diverse Target and Contribution Scheduling for Domain Generalizationcited by 10
  11. 2023Distilling Out-of-Distribution Robustness from Vision-Language Foundation Modelscited by 9
  12. 2024Prompt-Driven Contrastive Learning for Transferable Adversarial Attackscited by 9
28 more
  1. 2025When and How Does CLIP Enable Domain and Compositional Generalization?cited by 9
  2. 2024VLM-KD: Knowledge Distillation from VLM for Long-Tail Visual Recognitioncited by 7
  3. 2025Generalizing vision-language models to novel domains: A comprehensive surveycited by 7
  4. 2025CoPT: Unsupervised Domain Adaptive Segmentation using Domain-Agnostic Text Embeddingscited by 7
  5. 2024PromptTA: Prompt-driven Text Adapter for Source-free Domain Generalizationcited by 6
  6. 2025OSLoPrompt: Bridging Low-Supervision Challenges and Open-Set Domain Generalization in CLIPcited by 6
  7. 2025Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directionscited by 5
  8. 2024Tell, Don't Show!: Language Guidance Eases Transfer Across Domains in Images and Videoscited by 4
  9. 2024An Individual Identity-Driven Framework for Animal Re-Identificationcited by 3
  10. 2025Domain Generalization via Discrete Codebook Learningcited by 3
  11. 2024Transitive Vision-Language Prompt Learning for Domain Generalizationcited by 2
  12. 2024UMFC: Unsupervised Multi-Domain Feature Calibration for Vision-Language Modelscited by 2
  13. 2025Does CLIP perceive art the same way we do?cited by 2
  14. 2024Attention Head Purification: A New Perspective to Harness CLIP for Domain Generalizationcited by 1
  15. 2025Neural Antidote: Class-Wise Prompt Tuning for Purifying Backdoors in CLIPcited by 1
  16. 2025LangDA: Building Context-Awareness via Language for Domain Adaptive Semantic Segmentationcited by 1
  17. 2025FEDTAIL: Federated Long-Tailed Domain Generalization with Sharpness-Guided Gradient Matchingcited by 1
  18. 2025Generalizing Vision-Language Models with Dedicated Prompt Guidancecited by 1
  19. 2023Choosing Wisely and Learning Deeply: Selective Cross-Modality Distillation via CLIP for Domain Generalizationcited by 0
  20. 2024Semantically Guided Action Anticipationcited by 0
  21. 2025Generative Classifier for Domain Generalizationcited by 0
  22. 2025Towards Cross-Domain Multi-Targeted Adversarial Attackscited by 0
  23. 2025TRUST: Leveraging Text Robustness for Unsupervised Domain Adaptationcited by 0
  24. 2025All Centers Are at most a Few Tokens Apart: Knowledge Distillation with Domain Invariant Prompt Tuningcited by 0
  25. 2026SemAlign: Language Guided Semi-supervised Domain Generalizationcited by 0
  26. 2026PAND: Prompt-Aware Neighborhood Distillation for Lightweight Fine-Grained Visual Classificationcited by 0
  27. 2026DAIT: Distillation from Vision-Language Models to Lightweight Classifiers with Adaptive Intermediate Teacher Transfercited by 0
  28. 2026Domain Generalization via Text-Anchored Information Bottleneckcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Latent Domain Prompt Learning for Vision-Language Models
  2. 2023Strong but simple: A Baseline for Domain Generalized Dense Perception by CLIP-based Transfer Learning
  3. 2022Domain-Unified Prompt Representations for Source-Free Domain Generalization
  4. 2023Robust Fine-Tuning of Vision-Language Models for Domain Generalization
  5. 2023Distilling Large Vision-Language Model with Out-of-Distribution Generalizability
  6. 2026SHED: Style-Homogenized Embedding Alignment for Domain Generalization
  7. 2024Efficient and Long-Tailed Generalization for Pre-trained Vision-Language Model
  8. 2023A Simple Recipe for Language-guided Domain Generalized Segmentation