# Adversarially Adaptive Normalization for Single Domain Generalization

arXiv: https://arxiv.org/abs/2106.01899 · HTML: https://mlsanity.com/paper/2106.01899/

## Key facts

- Cited by 85 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 63 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 30 of 43)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [Generalizing to Unseen Domains via Adversarial Data Augmentation](https://mlsanity.com/paper/1805.12018/) (2018)
- [Domain Generalization by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1903.06864/) (2019)
- [Self-Challenging Improves Cross-Domain Generalization](https://mlsanity.com/paper/2007.02454/) (2020)
- [Certifying Some Distributional Robustness with Principled Adversarial Training](https://mlsanity.com/paper/1710.10571/) (2017)
- [Unified Deep Supervised Domain Adaptation and Generalization](https://mlsanity.com/paper/1709.10190/) (2017)
- [Learning to Learn Single Domain Generalization](https://mlsanity.com/paper/2003.13216/) (2020)
- [Learning to Generate Novel Domains for Domain Generalization](https://mlsanity.com/paper/2007.03304/) (2020)
- [Pretrained Transformers Improve Out-of-Distribution Robustness](https://mlsanity.com/paper/2004.06100/) (2020)
- [DropBlock: A regularization method for convolutional networks](https://mlsanity.com/paper/1810.12890/) (2018)
- [Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift](https://mlsanity.com/paper/2006.10963/) (2020)

## Built on by (most cited first, 30 of 85)

- [Domain Generalization by Mutual-Information Regularization with Pre-trained Models](https://mlsanity.com/paper/2203.10789/) (2022)
- [Exact Feature Distribution Matching for Arbitrary Style Transfer and Domain Generalization](https://mlsanity.com/paper/2203.07740/) (2022)
- [CLIP the Gap: A Single Domain Generalization Approach for Object Detection](https://mlsanity.com/paper/2301.05499/) (2023)
- [Adversarial Style Augmentation for Domain Generalized Urban-Scene Segmentation](https://mlsanity.com/paper/2207.04892/) (2022)
- [PromptStyler: Prompt-driven Style Generation for Source-free Domain Generalization](https://mlsanity.com/paper/2307.15199/) (2023)
- [Generalizable Cross-modality Medical Image Segmentation via Style Augmentation and Dual Normalization](https://mlsanity.com/paper/2112.11177/) (2021)
- [Meta-causal Learning for Single Domain Generalization](https://mlsanity.com/paper/2304.03709/) (2023)
- [PASTA: Proportional Amplitude Spectrum Training Augmentation for Syn-to-Real Domain Generalization](https://mlsanity.com/paper/2212.00979/) (2022)
- [Attention Consistency on Visual Corruptions for Single-Source Domain Generalization](https://mlsanity.com/paper/2204.13091/) (2022)
- [Progressive Random Convolutions for Single Domain Generalization](https://mlsanity.com/paper/2304.00424/) (2023)
- [Out-of-Distribution Representation Learning for Time Series Classification](https://mlsanity.com/paper/2209.07027/) (2022)
- [Domain and Content Adaptive Convolution based Multi-Source Domain Generalization for Medical Image Segmentation](https://mlsanity.com/paper/2109.05676/) (2021)
- [Improving Single Domain-Generalized Object Detection: A Focus on Diversification and Alignment](https://mlsanity.com/paper/2405.14497/) (2024)
- [Unbiased Faster R-CNN for Single-source Domain Generalized Object Detection](https://mlsanity.com/paper/2405.15225/) (2024)
- [ALOFT: A Lightweight MLP-like Architecture with Dynamic Low-frequency Transform for Domain Generalization](https://mlsanity.com/paper/2303.11674/) (2023)
- [SRCD: Semantic Reasoning with Compound Domains for Single-Domain Generalized Object Detection](https://mlsanity.com/paper/2307.01750/) (2023)
- [Decompose, Adjust, Compose: Effective Normalization by Playing with Frequency for Domain Generalization](https://mlsanity.com/paper/2303.02328/) (2023)
- [DIVERSIFY: A General Framework for Time Series Out-of-distribution Detection and Generalization](https://mlsanity.com/paper/2308.02282/) (2023)
- [Prompt-Driven Dynamic Object-Centric Learning for Single Domain Generalization](https://mlsanity.com/paper/2402.18447/) (2024)
- [Single-domain Generalization in Medical Image Segmentation via Test-time Adaptation from Shape Dictionary](https://mlsanity.com/paper/2206.14467/) (2022)
- [DomainDrop: Suppressing Domain-Sensitive Channels for Domain Generalization](https://mlsanity.com/paper/2308.10285/) (2023)
- [Contrastive ACE: Domain Generalization Through Alignment of Causal Mechanisms](https://mlsanity.com/paper/2106.00925/) (2021)
- [DomainAdaptor: A Novel Approach to Test-time Adaptation](https://mlsanity.com/paper/2308.10297/) (2023)
- [AADG: Automatic Augmentation for Domain Generalization on Retinal Image Segmentation](https://mlsanity.com/paper/2207.13249/) (2022)
- [Source-Free Domain Adaptation for Real-world Image Dehazing](https://mlsanity.com/paper/2207.06644/) (2022)
- [Learning Instance-Specific Adaptation for Cross-Domain Segmentation](https://mlsanity.com/paper/2203.16530/) (2022)
- [Deep transfer learning for image classification: a survey](https://mlsanity.com/paper/2205.09904/) (2022)
- [Normalization Enhances Generalization in Visual Reinforcement Learning](https://mlsanity.com/paper/2306.00656/) (2023)
- [Domain Generalization in Machine Learning Models for Wireless Communications: Concepts, State-of-the-Art, and Open Issues](https://mlsanity.com/paper/2303.08106/) (2023)
- [Fourier Test-time Adaptation with Multi-level Consistency for Robust Classification](https://mlsanity.com/paper/2306.02544/) (2023)

## Abstract (arXiv)

Single domain generalization aims to learn a model that performs well on many unseen domains with only one domain data for training. Existing works focus on studying the adversarial domain augmentation (ADA) to improve the model's generalization capability. The impact on domain generalization of the statistics of normalization layers is still underinvestigated. In this paper, we propose a generic normalization approach, adaptive standardization and rescaling normalization (ASR-Norm), to complement the missing part in previous works. ASR-Norm learns both the standardization and rescaling statistics via neural networks. This new form of normalization can be viewed as a generic form of the traditional normalizations. When trained with ADA, the statistics in ASR-Norm are learned to be adaptive to the data coming from different domains, and hence improves the model generalization performance across domains, especially on the target domain with large discrepancy from the source domain. The experimental results show that ASR-Norm can bring consistent improvement to the state-of-the-art ADA approaches by 1.6%, 2.7%, and 6.3% averagely on the Digits, CIFAR-10-C, and PACS benchmarks, respectively. As a generic tool, the improvement introduced by ASR-Norm is agnostic to the choice of ADA methods.