# MixStyle Neural Networks for Domain Generalization and Adaptation

arXiv: https://arxiv.org/abs/2107.02053 · HTML: https://mlsanity.com/paper/2107.02053/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- Builds on 70 of its 94 references that are in the map.
- First posted on arXiv 5 Jul 2021.

## Builds on (most cited first, 30 of 70)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)

## Built on by (most cited first, 30 of 64)

- [MADG: Margin-based Adversarial Learning for Domain Generalization](https://mlsanity.com/paper/2311.08503/) (2023)
- [DGMamba: Domain Generalization via Generalized State Space Model](https://mlsanity.com/paper/2404.07794/) (2024)
- [MVDG: A Unified Multi-view Framework for Domain Generalization](https://mlsanity.com/paper/2112.12329/) (2021)
- [Localized Adversarial Domain Generalization](https://mlsanity.com/paper/2205.04114/) (2022)
- [Towards Artificial General Intelligence (AGI) in the Internet of Things (IoT): Opportunities and Challenges](https://mlsanity.com/paper/2309.07438/) (2023)
- [Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution Generalization](https://mlsanity.com/paper/2310.16391/) (2023)
- [DASK: Distribution Rehearsing via Adaptive Style Kernel Learning for Exemplar-Free Lifelong Person Re-Identification](https://mlsanity.com/paper/2412.09224/) (2024)
- [Tackling Long-Tailed Category Distribution Under Domain Shifts](https://mlsanity.com/paper/2207.10150/) (2022)
- [TFS-ViT: Token-Level Feature Stylization for Domain Generalization](https://mlsanity.com/paper/2303.15698/) (2023)
- [Diverse Target and Contribution Scheduling for Domain Generalization](https://mlsanity.com/paper/2309.16460/) (2023)
- [StyleAM: Perception-Oriented Unsupervised Domain Adaption for Non-reference Image Quality Assessment](https://mlsanity.com/paper/2207.14489/) (2022)
- [Moderately Distributional Exploration for Domain Generalization](https://mlsanity.com/paper/2304.13976/) (2023)
- [Generative Medical Segmentation](https://mlsanity.com/paper/2403.18198/) (2024)
- [UniDemoir'e: Towards Universal Image Demoir'eing with Data Generation and Synthesis](https://mlsanity.com/paper/2502.06324/) (2025)
- [CauDR: A Causality-inspired Domain Generalization Framework for Fundus-based Diabetic Retinopathy Grading](https://mlsanity.com/paper/2309.15493/) (2023)
- [SynFacePAD 2023: Competition on Face Presentation Attack Detection Based on Privacy-aware Synthetic Training Data](https://mlsanity.com/paper/2311.05336/) (2023)
- [Domain Generalization for 6D Pose Estimation Through NeRF-based Image Synthesis](https://mlsanity.com/paper/2407.10762/) (2024)
- [Exploring Flat Minima for Domain Generalization with Large Learning Rates](https://mlsanity.com/paper/2309.06337/) (2023)
- [START: A Generalized State Space Model with Saliency-Driven Token-Aware Transformation](https://mlsanity.com/paper/2410.16020/) (2024)
- [Align the GAP: Prior-based Unified Multi-Task Remote Physiological Measurement Framework For Domain Generalization and Personalization](https://mlsanity.com/paper/2506.16160/) (2025)
- [DomainVerse: A Benchmark Towards Real-World Distribution Shifts For Tuning-Free Adaptive Domain Generalization](https://mlsanity.com/paper/2403.02714/) (2024)
- [SETA: Semantic-Aware Token Augmentation for Domain Generalization](https://mlsanity.com/paper/2403.11792/) (2024)
- [Decomposed Distribution Matching in Dataset Condensation](https://mlsanity.com/paper/2412.04748/) (2024)
- [Domain Generalization via Discrete Codebook Learning](https://mlsanity.com/paper/2504.06572/) (2025)
- [Bridging the Source-to-target Gap for Cross-domain Person Re-Identification with Intermediate Domains](https://mlsanity.com/paper/2203.01682/) (2022)
- [Robust Channel Learning for Large-Scale Radio Speaker Verification](https://mlsanity.com/paper/2406.10956/) (2024)
- [Domain Expansion and Boundary Growth for Open-Set Single-Source Domain Generalization](https://mlsanity.com/paper/2411.02920/) (2024)
- [TIDE: Training Locally Interpretable Domain Generalization Models Enables Test-time Correction](https://mlsanity.com/paper/2411.16788/) (2024)
- [GeneralizeFormer: Layer-Adaptive Model Generation across Test-Time Distribution Shifts](https://mlsanity.com/paper/2502.12195/) (2025)
- [Let Synthetic Data Shine: Domain Reassembly and Soft-Fusion for Single Domain Generalization](https://mlsanity.com/paper/2503.13617/) (2025)

## Abstract (arXiv)

Neural networks do not generalize well to unseen data with domain shifts -- a longstanding problem in machine learning and AI. To overcome the problem, we propose MixStyle, a simple plug-and-play, parameter-free module that can improve domain generalization performance without the need to collect more data or increase model capacity. The design of MixStyle is simple: it mixes the feature statistics of two random instances in a single forward pass during training. The idea is grounded by the finding from recent style transfer research that feature statistics capture image style information, which essentially defines visual domains. Therefore, mixing feature statistics can be seen as an efficient way to synthesize new domains in the feature space, thus achieving data augmentation. MixStyle is easy to implement with a few lines of code, does not require modification to training objectives, and can fit a variety of learning paradigms including supervised domain generalization, semi-supervised domain generalization, and unsupervised domain adaptation. Our experiments show that MixStyle can significantly boost out-of-distribution generalization performance across a wide range of tasks including image recognition, instance retrieval and reinforcement learning.