# Uncertainty Modeling for Out-of-Distribution Generalization

arXiv: https://arxiv.org/abs/2202.03958 · HTML: https://mlsanity.com/paper/2202.03958/

## Key facts

- Cited by 125 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 57 references that are in the map.
- First posted on arXiv 8 Feb 2022.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [Domain Generalization with MixStyle](https://mlsanity.com/paper/2104.02008/) (2021)
- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Self-Challenging Improves Cross-Domain Generalization](https://mlsanity.com/paper/2007.02454/) (2020)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net](https://mlsanity.com/paper/1807.09441/) (2018)
- [Learning to Generate Novel Domains for Domain Generalization](https://mlsanity.com/paper/2007.03304/) (2020)
- [Generalizing Across Domains via Cross-Gradient Training](https://mlsanity.com/paper/1804.10745/) (2018)
- [Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification](https://mlsanity.com/paper/2001.01526/) (2020)
- [RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening](https://mlsanity.com/paper/2103.15597/) (2021)
- [Probabilistic Face Embeddings](https://mlsanity.com/paper/1904.09658/) (2019)
- [Learning to Optimize Domain Specific Normalization for Domain Generalization](https://mlsanity.com/paper/1907.04275/) (2019)
- [Domain Adaptive Ensemble Learning](https://mlsanity.com/paper/2003.07325/) (2020)
- [Classes Matter: A Fine-grained Adversarial Approach to Cross-domain Semantic Segmentation](https://mlsanity.com/paper/2007.09222/) (2020)

## Built on by (most cited first, 30 of 125)

- [EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularization](https://mlsanity.com/paper/2303.01904/) (2023)
- [Dual Memory Units with Uncertainty Regulation for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2302.05160/) (2023)
- [StyleAdv: Meta Style Adversarial Training for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2302.09309/) (2023)
- [Improved Test-Time Adaptation for Domain Generalization](https://mlsanity.com/paper/2304.04494/) (2023)
- [BayeSeg: Bayesian Modeling for Medical Image Segmentation with Interpretable Generalizability](https://mlsanity.com/paper/2303.01710/) (2023)
- [Feature Alignment and Uniformity for Test Time Adaptation](https://mlsanity.com/paper/2303.10902/) (2023)
- [PASTA: Proportional Amplitude Spectrum Training Augmentation for Syn-to-Real Domain Generalization](https://mlsanity.com/paper/2212.00979/) (2022)
- [Towards Open-Set Test-Time Adaptation Utilizing the Wisdom of Crowds in Entropy Minimization](https://mlsanity.com/paper/2308.06879/) (2023)
- [A survey of synthetic data augmentation methods in computer vision](https://mlsanity.com/paper/2403.10075/) (2024)
- [Enhance Vision-Language Alignment with Noise](https://mlsanity.com/paper/2412.10817/) (2024)
- [Prompting Diffusion Representations for Cross-Domain Semantic Segmentation](https://mlsanity.com/paper/2307.02138/) (2023)
- [Attention Diversification for Domain Generalization](https://mlsanity.com/paper/2210.04206/) (2022)
- [ALOFT: A Lightweight MLP-like Architecture with Dynamic Low-frequency Transform for Domain Generalization](https://mlsanity.com/paper/2303.11674/) (2023)
- [SRCD: Semantic Reasoning with Compound Domains for Single-Domain Generalized Object Detection](https://mlsanity.com/paper/2307.01750/) (2023)
- [MaxStyle: Adversarial Style Composition for Robust Medical Image Segmentation](https://mlsanity.com/paper/2206.01737/) (2022)
- [Delving into Semantic Scale Imbalance](https://mlsanity.com/paper/2212.14613/) (2022)
- [FedFA: Federated Feature Augmentation](https://mlsanity.com/paper/2301.12995/) (2023)
- [StableFDG: Style and Attention Based Learning for Federated Domain Generalization](https://mlsanity.com/paper/2311.00227/) (2023)
- [Mix and Reason: Reasoning over Semantic Topology with Data Mixing for Domain Generalization](https://mlsanity.com/paper/2210.07571/) (2022)
- [Intra-Source Style Augmentation for Improved Domain Generalization](https://mlsanity.com/paper/2210.10175/) (2022)
- [Adversarial Style Augmentation for Domain Generalization](https://mlsanity.com/paper/2301.12643/) (2023)
- [FisherTune: Fisher-Guided Robust Tuning of Vision Foundation Models for Domain Generalized Segmentation](https://mlsanity.com/paper/2503.17940/) (2025)
- [Treasure in Distribution: A Domain Randomization based Multi-Source Domain Generalization for 2D Medical Image Segmentation](https://mlsanity.com/paper/2305.19949/) (2023)
- [Modality-Agnostic Debiasing for Single Domain Generalization](https://mlsanity.com/paper/2303.07123/) (2023)
- [Domain-Rectifying Adapter for Cross-Domain Few-Shot Segmentation](https://mlsanity.com/paper/2404.10322/) (2024)
- [Deep transfer learning for image classification: a survey](https://mlsanity.com/paper/2205.09904/) (2022)
- [The RoboDepth Challenge: Methods and Advancements Towards Robust Depth Estimation](https://mlsanity.com/paper/2307.15061/) (2023)
- [Test-Time Style Shifting: Handling Arbitrary Styles in Domain Generalization](https://mlsanity.com/paper/2306.04911/) (2023)
- [Domain Invariant Masked Autoencoders for Self-supervised Learning from Multi-domains](https://mlsanity.com/paper/2205.04771/) (2022)
- [Domain Generalization with Correlated Style Uncertainty](https://mlsanity.com/paper/2212.09950/) (2022)

## Abstract (arXiv)

Though remarkable progress has been achieved in various vision tasks, deep neural networks still suffer obvious performance degradation when tested in out-of-distribution scenarios. We argue that the feature statistics (mean and standard deviation), which carry the domain characteristics of the training data, can be properly manipulated to improve the generalization ability of deep learning models. Common methods often consider the feature statistics as deterministic values measured from the learned features and do not explicitly consider the uncertain statistics discrepancy caused by potential domain shifts during testing. In this paper, we improve the network generalization ability by modeling the uncertainty of domain shifts with synthesized feature statistics during training. Specifically, we hypothesize that the feature statistic, after considering the potential uncertainties, follows a multivariate Gaussian distribution. Hence, each feature statistic is no longer a deterministic value, but a probabilistic point with diverse distribution possibilities. With the uncertain feature statistics, the models can be trained to alleviate the domain perturbations and achieve better robustness against potential domain shifts. Our method can be readily integrated into networks without additional parameters. Extensive experiments demonstrate that our proposed method consistently improves the network generalization ability on multiple vision tasks, including image classification, semantic segmentation, and instance retrieval. The code can be available at https://github.com/lixiaotong97/DSU.