# A Simple Recipe for Language-guided Domain Generalized Segmentation

arXiv: https://arxiv.org/abs/2311.17922 · HTML: https://mlsanity.com/paper/2311.17922/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 75 references that are in the map.
- First posted on arXiv 29 Nov 2023.

## Builds on (most cited first, 30 of 58)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [Open-vocabulary Object Detection via Vision and Language Knowledge Distillation](https://mlsanity.com/paper/2104.13921/) (2021)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Robust fine-tuning of zero-shot models](https://mlsanity.com/paper/2109.01903/) (2021)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [Language-driven Semantic Segmentation](https://mlsanity.com/paper/2201.03546/) (2022)
- [ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1811.12833/) (2018)
- [Extract Free Dense Labels from CLIP](https://mlsanity.com/paper/2112.01071/) (2021)
- [LiT: Zero-Shot Transfer with Locked-image text Tuning](https://mlsanity.com/paper/2111.07991/) (2021)

## Built on by (most cited first, 28 of 28)

- [Stronger, Fewer, & Superior: Harnessing Vision Foundation Models for Domain Generalized Semantic Segmentation](https://mlsanity.com/paper/2312.04265/) (2023)
- [SoMA: Singular Value Decomposed Minor Components Adaptation for Domain Generalizable Representation Learning](https://mlsanity.com/paper/2412.04077/) (2024)
- [Mamba as a Bridge: Where Vision Foundation Models Meet Vision Language Models for Domain-Generalized Semantic Segmentation](https://mlsanity.com/paper/2504.03193/) (2025)
- [Textual Query-Driven Mask Transformer for Domain Generalized Segmentation](https://mlsanity.com/paper/2407.09033/) (2024)
- [Strong but simple: A Baseline for Domain Generalized Dense Perception by CLIP-based Transfer Learning](https://mlsanity.com/paper/2312.02021/) (2023)
- [FisherTune: Fisher-Guided Robust Tuning of Vision Foundation Models for Domain Generalized Segmentation](https://mlsanity.com/paper/2503.17940/) (2025)
- [Large Self-Supervised Models Bridge the Gap in Domain Adaptive Object Detection](https://mlsanity.com/paper/2503.23220/) (2025)
- [Exploring Semantic Consistency and Style Diversity for Domain Generalized Semantic Segmentation](https://mlsanity.com/paper/2412.12050/) (2024)
- [The BRAVO Semantic Segmentation Challenge Results in UNCV2024](https://mlsanity.com/paper/2409.15107/) (2024)
- [Exploiting Domain Properties in Language-Driven Domain Generalization for Semantic Segmentation](https://mlsanity.com/paper/2512.03508/) (2025)
- [SemiDAViL: Semi-supervised Domain Adaptation with Vision-Language Guidance for Semantic Segmentation](https://mlsanity.com/paper/2504.06389/) (2025)
- [LatteCLIP: Unsupervised CLIP Fine-Tuning via LMM-Synthetic Texts](https://mlsanity.com/paper/2410.08211/) (2024)
- [Rein++: Efficient Generalization and Adaptation for Semantic Segmentation with Vision Foundation Models](https://mlsanity.com/paper/2508.01667/) (2025)
- [Domain Adaptation with a Single Vision-Language Embedding](https://mlsanity.com/paper/2410.21361/) (2024)
- [A Study on Unsupervised Domain Adaptation for Semantic Segmentation in the Era of Vision-Language Models](https://mlsanity.com/paper/2411.16407/) (2024)
- [Set Pivot Learning: Redefining Generalized Segmentation with Vision Foundation Models](https://mlsanity.com/paper/2508.01582/) (2025)
- [Multi-Granularity Feature Calibration via VFM for Domain Generalized Semantic Segmentation](https://mlsanity.com/paper/2508.03007/) (2025)
- [IELDG: Suppressing Domain-Specific Noise with Inverse Evolution Layers for Domain Generalized Semantic Segmentation](https://mlsanity.com/paper/2508.19604/) (2025)
- [Domain Generalization for Semantic Segmentation: A Survey](https://mlsanity.com/paper/2510.03540/) (2025)
- [Do We Need Perfect Data? Leveraging Noise for Domain Generalized Segmentation](https://mlsanity.com/paper/2511.22948/) (2025)
- [SAGE: Style-Adaptive Generalization for Privacy-Constrained Semantic Segmentation Across Domains](https://mlsanity.com/paper/2512.02369/) (2025)
- [OT-Drive: Out-of-Distribution Off-Road Traversable Area Segmentation via Optimal Transport](https://mlsanity.com/paper/2601.09952/) (2026)
- [RecycleLoRA: Rank-Revealing QR-Based Dual-LoRA Subspace Adaptation for Domain Generalized Semantic Segmentation](https://mlsanity.com/paper/2603.28142/) (2026)
- [Dual-Foundation Models for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2605.03365/) (2026)
- [MQAdapter: Multi-Modal Quantum Adapter for Coarse-to-Fine VLM Fine-tuning](https://mlsanity.com/paper/2607.12418/) (2026)
- [LASA: Language-and-Source-Anchored Alignment for Domain Generalized Semantic Segmentation](https://mlsanity.com/paper/2608.08805/) (2026)
- [Hierarchical Prompt Injector for Domain Generalization Segmentation](https://mlsanity.com/paper/2609.05864/) (2026)
- [Sparse All-Layer Connector for Domain Generalised Semantic Segmentation](https://mlsanity.com/paper/2610.15660/) (2026)

## Abstract (arXiv)

Generalization to new domains not seen during training is one of the long-standing challenges in deploying neural networks in real-world applications. Existing generalization techniques either necessitate external images for augmentation, and/or aim at learning invariant representations by imposing various alignment constraints. Large-scale pretraining has recently shown promising generalization capabilities, along with the potential of binding different modalities. For instance, the advent of vision-language models like CLIP has opened the doorway for vision models to exploit the textual modality. In this paper, we introduce a simple framework for generalizing semantic segmentation networks by employing language as the source of randomization. Our recipe comprises three key ingredients: (i) the preservation of the intrinsic CLIP robustness through minimal fine-tuning, (ii) language-driven local style augmentation, and (iii) randomization by locally mixing the source and augmented styles during training. Extensive experiments report state-of-the-art results on various generalization benchmarks. Code is accessible at https://github.com/astra-vision/FAMix .