# Dual Stage Stylization Modulation for Domain Generalized Semantic Segmentation

arXiv: https://arxiv.org/abs/2304.09347 · HTML: https://mlsanity.com/paper/2304.09347/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 39 of its 59 references that are in the map.
- First posted on arXiv 18 Apr 2023.

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1811.12833/) (2018)
- [Domain Generalization with MixStyle](https://mlsanity.com/paper/2104.02008/) (2021)
- [Generalizing to Unseen Domains via Adversarial Data Augmentation](https://mlsanity.com/paper/1805.12018/) (2018)
- [Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform](https://mlsanity.com/paper/1804.02815/) (2018)
- [Learning to Learn Single Domain Generalization](https://mlsanity.com/paper/2003.13216/) (2020)
- [Taking A Closer Look at Domain Shift: Category-level Adversaries for Semantics Consistent Domain Adaptation](https://mlsanity.com/paper/1809.09478/) (2018)
- [A Fourier-based Framework for Domain Generalization](https://mlsanity.com/paper/2105.11120/) (2021)
- [Structured Knowledge Distillation for Dense Prediction](https://mlsanity.com/paper/1903.04197/) (2019)
- [Generalizing Across Domains via Cross-Gradient Training](https://mlsanity.com/paper/1804.10745/) (2018)
- [Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Data](https://mlsanity.com/paper/1909.00889/) (2019)
- [Domain Adaptation for Semantic Segmentation with Maximum Squares Loss](https://mlsanity.com/paper/1909.13589/) (2019)
- [Adversarial Feature Hallucination Networks for Few-Shot Learning](https://mlsanity.com/paper/2003.13193/) (2020)
- [Progressive Domain Expansion Network for Single Domain Generalization](https://mlsanity.com/paper/2103.16050/) (2021)

## Abstract (arXiv)

Obtaining sufficient labeled data for training deep models is often challenging in real-life applications. To address this issue, we propose a novel solution for single-source domain generalized semantic segmentation. Recent approaches have explored data diversity enhancement using hallucination techniques. However, excessive hallucination can degrade performance, particularly for imbalanced datasets. As shown in our experiments, minority classes are more susceptible to performance reduction due to hallucination compared to majority classes. To tackle this challenge, we introduce a dual-stage Feature Transform (dFT) layer within the Adversarial Semantic Hallucination+ (ASH+) framework. The ASH+ framework performs a dual-stage manipulation of hallucination strength. By leveraging semantic information for each pixel, our approach adaptively adjusts the pixel-wise hallucination strength, thus providing fine-grained control over hallucination. We validate the effectiveness of our proposed method through comprehensive experiments on publicly available semantic segmentation benchmark datasets (Cityscapes and SYNTHIA). Quantitative and qualitative comparisons demonstrate that our approach is competitive with state-of-the-art methods for the Cityscapes dataset and surpasses existing solutions for the SYNTHIA dataset. Code for our framework will be made readily available to the research community.