# Grounding Visual Representations with Texts for Domain Generalization

arXiv: https://arxiv.org/abs/2207.10285 · HTML: https://mlsanity.com/paper/2207.10285/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 63 references that are in the map.
- First posted on arXiv 21 Jul 2022.

## Builds on (most cited first, 30 of 41)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Domain Generalization with MixStyle](https://mlsanity.com/paper/2104.02008/) (2021)
- [Self-Challenging Improves Cross-Domain Generalization](https://mlsanity.com/paper/2007.02454/) (2020)
- [Certifying Some Distributional Robustness with Principled Adversarial Training](https://mlsanity.com/paper/1710.10571/) (2017)
- [Gradient Starvation: A Learning Proclivity in Neural Networks](https://mlsanity.com/paper/2011.09468/) (2020)
- [Adversarial Domain Adaptation with Domain Mixup](https://mlsanity.com/paper/1912.01805/) (2019)
- [SelfReg: Self-supervised Contrastive Regularization for Domain Generalization](https://mlsanity.com/paper/2104.09841/) (2021)

## Built on by (most cited first, 30 of 30)

- [PromptStyler: Prompt-driven Style Generation for Source-free Domain Generalization](https://mlsanity.com/paper/2307.15199/) (2023)
- [Read-only Prompt Optimization for Vision-Language Few-shot Learning](https://mlsanity.com/paper/2308.14960/) (2023)
- [What can a cook in Italy teach a mechanic in India? Action Recognition Generalisation Over Scenarios and Locations](https://mlsanity.com/paper/2306.08713/) (2023)
- [Domain-Unified Prompt Representations for Source-Free Domain Generalization](https://mlsanity.com/paper/2209.14926/) (2022)
- [DomainDrop: Suppressing Domain-Sensitive Channels for Domain Generalization](https://mlsanity.com/paper/2308.10285/) (2023)
- [HYPO: Hyperspherical Out-of-Distribution Generalization](https://mlsanity.com/paper/2402.07785/) (2024)
- [CIFAR-10-Warehouse: Broad and More Realistic Testbeds in Model Generalization Analysis](https://mlsanity.com/paper/2310.04414/) (2023)
- [Vision-Language Model for Object Detection and Segmentation: A Review and Evaluation](https://mlsanity.com/paper/2504.09480/) (2025)
- [Gradient Estimation for Unseen Domain Risk Minimization with Pre-Trained Models](https://mlsanity.com/paper/2302.01497/) (2023)
- [NormAUG: Normalization-guided Augmentation for Domain Generalization](https://mlsanity.com/paper/2307.13492/) (2023)
- [Improving Large Language Model (LLM) fidelity through context-aware grounding: A systematic approach to reliability and veracity](https://mlsanity.com/paper/2408.04023/) (2024)
- [AHA: Human-Assisted Out-of-Distribution Generalization and Detection](https://mlsanity.com/paper/2410.08000/) (2024)
- [CoPT: Unsupervised Domain Adaptive Segmentation using Domain-Agnostic Text Embeddings](https://mlsanity.com/paper/2507.07125/) (2025)
- [DPStyler: Dynamic PromptStyler for Source-Free Domain Generalization](https://mlsanity.com/paper/2403.16697/) (2024)
- [Out-of-Distribution Learning with Human Feedback](https://mlsanity.com/paper/2408.07772/) (2024)
- [Learning Generalizable Models via Disentangling Spurious and Enhancing Potential Correlations](https://mlsanity.com/paper/2401.05752/) (2024)
- [Predicting Class Distribution Shift for Reliable Domain Adaptive Object Detection](https://mlsanity.com/paper/2302.06039/) (2023)
- [Beyond Finite Data: Towards Data-free Out-of-distribution Generalization via Extrapolation](https://mlsanity.com/paper/2403.05523/) (2024)
- [Tell, Don't Show!: Language Guidance Eases Transfer Across Domains in Images and Videos](https://mlsanity.com/paper/2403.05535/) (2024)
- [On-Device Domain Generalization](https://mlsanity.com/paper/2209.07521/) (2022)
- [Multimodal Unsupervised Domain Generalization by Retrieving Across the Modality Gap](https://mlsanity.com/paper/2402.04416/) (2024)
- [SETA: Semantic-Aware Token Augmentation for Domain Generalization](https://mlsanity.com/paper/2403.11792/) (2024)
- [DoubleAUG: Single-domain Generalized Object Detector in Urban via Color Perturbation and Dual-style Memory](https://mlsanity.com/paper/2311.13198/) (2023)
- [TLDR: Text Based Last-layer Retraining for Debiasing Image Classifiers](https://mlsanity.com/paper/2311.18291/) (2023)
- [Integrating Audio Narrations to Strengthen Domain Generalization in Multimodal First-Person Action Recognition](https://mlsanity.com/paper/2409.09611/) (2024)
- [QueryAdapter: Rapid Adaptation of Vision-Language Models in Response to Natural Language Queries](https://mlsanity.com/paper/2502.18735/) (2025)
- [Clustering-based Image-Text Graph Matching for Domain Generalization](https://mlsanity.com/paper/2310.02692/) (2023)
- [StylePrompter: Enhancing Domain Generalization with Test-Time Style Priors](https://mlsanity.com/paper/2408.09138/) (2024)
- [Semantically Guided Action Anticipation](https://mlsanity.com/paper/2411.15557/) (2024)
- [CRISP: Compositional Relations as Invariant Structural Priors for Domain Generalization](https://mlsanity.com/paper/2605.06043/) (2026)

## Abstract (arXiv)

Reducing the representational discrepancy between source and target domains is a key component to maximize the model generalization. In this work, we advocate for leveraging natural language supervision for the domain generalization task. We introduce two modules to ground visual representations with texts containing typical reasoning of humans: (1) Visual and Textual Joint Embedder and (2) Textual Explanation Generator. The former learns the image-text joint embedding space where we can ground high-level class-discriminative information into the model. The latter leverages an explainable model and generates explanations justifying the rationale behind its decision. To the best of our knowledge, this is the first work to leverage the vision-and-language cross-modality approach for the domain generalization task. Our experiments with a newly created CUB-DG benchmark dataset demonstrate that cross-modality supervision can be successfully used to ground domain-invariant visual representations and improve the model generalization. Furthermore, in the large-scale DomainBed benchmark, our proposed method achieves state-of-the-art results and ranks 1st in average performance for five multi-domain datasets. The dataset and codes are available at https://github.com/mswzeus/GVRT.