# A Bit More Bayesian: Domain-Invariant Learning with Uncertainty

arXiv: https://arxiv.org/abs/2105.04030 · HTML: https://mlsanity.com/paper/2105.04030/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 54 references that are in the map.
- First posted on arXiv 9 May 2021.

## Builds on (most cited first, 30 of 43)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Domain Generalization with MixStyle](https://mlsanity.com/paper/2104.02008/) (2021)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [Generalizing to Unseen Domains via Adversarial Data Augmentation](https://mlsanity.com/paper/1805.12018/) (2018)
- [Domain Generalization by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1903.06864/) (2019)
- [Self-Challenging Improves Cross-Domain Generalization](https://mlsanity.com/paper/2007.02454/) (2020)
- [Domain Generalization via Model-Agnostic Learning of Semantic Features](https://mlsanity.com/paper/1910.13580/) (2019)
- [Variational Continual Learning](https://mlsanity.com/paper/1710.10628/) (2017)
- [Unified Deep Supervised Domain Adaptation and Generalization](https://mlsanity.com/paper/1709.10190/) (2017)
- [Learning to Learn Single Domain Generalization](https://mlsanity.com/paper/2003.13216/) (2020)
- [Learning to Generate Novel Domains for Domain Generalization](https://mlsanity.com/paper/2007.03304/) (2020)
- [Generalizing Across Domains via Cross-Gradient Training](https://mlsanity.com/paper/1804.10745/) (2018)
- [Domain Generalization for Object Recognition with Multi-task Autoencoders](https://mlsanity.com/paper/1508.07680/) (2015)
- [Episodic Training for Domain Generalization](https://mlsanity.com/paper/1902.00113/) (2019)
- [Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks](https://mlsanity.com/paper/2002.10118/) (2020)

## Built on by (most cited first, 27 of 27)

- [Causality Inspired Representation Learning for Domain Generalization](https://mlsanity.com/paper/2203.14237/) (2022)
- [Improving Single Domain-Generalized Object Detection: A Focus on Diversification and Alignment](https://mlsanity.com/paper/2405.14497/) (2024)
- [Learning to Generalize across Domains on Single Test Samples](https://mlsanity.com/paper/2202.08045/) (2022)
- [Any-Shift Prompting for Generalization over Distributions](https://mlsanity.com/paper/2402.10099/) (2024)
- [Adversarial Bayesian Augmentation for Single-Source Domain Generalization](https://mlsanity.com/paper/2307.09520/) (2023)
- [Domain-Indexing Variational Bayes: Interpretable Domain Index for Domain Adaptation](https://mlsanity.com/paper/2302.02561/) (2023)
- [Variational Model Perturbation for Source-Free Domain Adaptation](https://mlsanity.com/paper/2210.10378/) (2022)
- [Hierarchical Variational Memory for Few-shot Learning Across Domains](https://mlsanity.com/paper/2112.08181/) (2021)
- [Energy-Based Test Sample Adaptation for Domain Generalization](https://mlsanity.com/paper/2302.11215/) (2023)
- [A General Framework for quantifying Aleatoric and Epistemic uncertainty in Graph Neural Networks](https://mlsanity.com/paper/2205.09968/) (2022)
- [A Simple Yet Effective Strategy to Robustify the Meta Learning Paradigm](https://mlsanity.com/paper/2310.00708/) (2023)
- [Causal Inference via Style Transfer for Out-of-distribution Generalisation](https://mlsanity.com/paper/2212.03063/) (2022)
- [Moderately Distributional Exploration for Domain Generalization](https://mlsanity.com/paper/2304.13976/) (2023)
- [Domain Generalization with Small Data](https://mlsanity.com/paper/2402.06150/) (2024)
- [QUOTA: Quantifying Objects with Text-to-Image Models for Any Domain](https://mlsanity.com/paper/2411.19534/) (2024)
- [Three approaches to facilitate DNN generalization to objects in out-of-distribution orientations and illuminations](https://mlsanity.com/paper/2111.00131/) (2021)
- [TIDE: Training Locally Interpretable Domain Generalization Models Enables Test-time Correction](https://mlsanity.com/paper/2411.16788/) (2024)
- [GeneralizeFormer: Layer-Adaptive Model Generation across Test-Time Distribution Shifts](https://mlsanity.com/paper/2502.12195/) (2025)
- [Calibration-based Dual Prototypical Contrastive Learning Approach for Domain Generalization Semantic Segmentation](https://mlsanity.com/paper/2309.14282/) (2023)
- [Bayesian Domain Invariant Learning via Posterior Generalization of Parameter Distributions](https://mlsanity.com/paper/2310.16277/) (2023)
- [R'enyi Neural Processes](https://mlsanity.com/paper/2405.15991/) (2024)
- [Towards Robust Uncertainty-Aware Incomplete Multi-View Classification](https://mlsanity.com/paper/2409.06270/) (2024)
- [VirDA: Reusing Backbone for Unsupervised Domain Adaptation with Visual Reprogramming](https://mlsanity.com/paper/2510.01660/) (2025)
- [Association Graph Learning for Multi-Task Classification with Category Shifts](https://mlsanity.com/paper/2210.04637/) (2022)
- [Be Bayesian by Attachments to Catch More Uncertainty](https://mlsanity.com/paper/2310.13027/) (2023)
- [Causal Understanding For Video Question Answering](https://mlsanity.com/paper/2407.20257/) (2024)
- [PEdger++: Practical Edge Detection via Assembling Cross Information](https://mlsanity.com/paper/2508.11961/) (2025)

## Abstract (arXiv)

Domain generalization is challenging due to the domain shift and the uncertainty caused by the inaccessibility of target domain data. In this paper, we address both challenges with a probabilistic framework based on variational Bayesian inference, by incorporating uncertainty into neural network weights. We couple domain invariance in a probabilistic formula with the variational Bayesian inference. This enables us to explore domain-invariant learning in a principled way. Specifically, we derive domain-invariant representations and classifiers, which are jointly established in a two-layer Bayesian neural network. We empirically demonstrate the effectiveness of our proposal on four widely used cross-domain visual recognition benchmarks. Ablation studies validate the synergistic benefits of our Bayesian treatment when jointly learning domain-invariant representations and classifiers for domain generalization. Further, our method consistently delivers state-of-the-art mean accuracy on all benchmarks.