# Improving Domain Generalization with Domain Relations

arXiv: https://arxiv.org/abs/2302.02609 · HTML: https://mlsanity.com/paper/2302.02609/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 67 references that are in the map.
- First posted on arXiv 6 Feb 2023.

## Builds on (most cited first, 30 of 51)

- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Language Modeling with Gated Convolutional Networks](https://mlsanity.com/paper/1612.08083/) (2016)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Continual Test-Time Domain Adaptation](https://mlsanity.com/paper/2203.13591/) (2022)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [Self-Challenging Improves Cross-Domain Generalization](https://mlsanity.com/paper/2007.02454/) (2020)
- [MEMO: Test Time Robustness via Adaptation and Augmentation](https://mlsanity.com/paper/2110.09506/) (2021)
- [Learning Factored Representations in a Deep Mixture of Experts](https://mlsanity.com/paper/1312.4314/) (2013)
- [Adversarial Domain Adaptation with Domain Mixup](https://mlsanity.com/paper/1912.01805/) (2019)
- [Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Data](https://mlsanity.com/paper/1909.00889/) (2019)
- [Deep Domain-Adversarial Image Generation for Domain Generalisation](https://mlsanity.com/paper/2003.06054/) (2020)
- [Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.06607/) (2021)
- [Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings](https://mlsanity.com/paper/2006.13009/) (2020)
- [Improving Out-of-Distribution Robustness via Selective Augmentation](https://mlsanity.com/paper/2201.00299/) (2022)
- [SelfReg: Self-supervised Contrastive Regularization for Domain Generalization](https://mlsanity.com/paper/2104.09841/) (2021)
- [Improve Unsupervised Domain Adaptation with Mixup Training](https://mlsanity.com/paper/2001.00677/) (2020)
- [Geometric Dataset Distances via Optimal Transport](https://mlsanity.com/paper/2002.02923/) (2020)
- [Open Domain Generalization with Domain-Augmented Meta-Learning](https://mlsanity.com/paper/2104.03620/) (2021)

## Built on by (most cited first, 14 of 14)

- [CrossEarth: Geospatial Vision Foundation Model for Domain Generalizable Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2410.22629/) (2024)
- [Symmetry-Informed Geometric Representation for Molecules, Proteins, and Crystalline Materials](https://mlsanity.com/paper/2306.09375/) (2023)
- [GraphMETRO: Mitigating Complex Graph Distribution Shifts via Mixture of Aligned Experts](https://mlsanity.com/paper/2312.04693/) (2023)
- [Confidence-Based Model Selection: When to Take Shortcuts for Subpopulation Shifts](https://mlsanity.com/paper/2306.11120/) (2023)
- [Hybrid-TTA: Continual Test-time Adaptation via Dynamic Domain Shift Detection](https://mlsanity.com/paper/2409.08566/) (2024)
- [From Invariant Representations to Invariant Data: Provable Robustness to Spurious Correlations via Noisy Counterfactual Matching](https://mlsanity.com/paper/2505.24843/) (2025)
- [Domain Generalization Under Posterior Drift](https://mlsanity.com/paper/2510.04441/) (2025)
- [CoDEx: Combining Domain Expertise for Spatial Generalization in Satellite Image Analysis](https://mlsanity.com/paper/2504.19737/) (2025)
- [Reliable OOD Virtual Screening with Extrapolatory Pseudo-Label Matching](https://mlsanity.com/paper/2406.01825/) (2024)
- [Latent Domain Modeling Improves Robustness to Geographic Shifts](https://mlsanity.com/paper/2503.02036/) (2025)
- [OT on the Map: Quantifying Domain Shifts in Geographic Space](https://mlsanity.com/paper/2604.16220/) (2026)
- [SemAlign: Language Guided Semi-supervised Domain Generalization](https://mlsanity.com/paper/2601.11724/) (2026)
- [Language-Induced Priors for Domain Adaptation](https://mlsanity.com/paper/2605.14301/) (2026)
- [Learning Subset-Shared Invariances for Domain Generalization with Mixture-of-Experts](https://mlsanity.com/paper/2606.25665/) (2026)

## Abstract (arXiv)

Distribution shift presents a significant challenge in machine learning, where models often underperform during the test stage when faced with a different distribution than the one they were trained on. This paper focuses on domain shifts, which occur when the model is applied to new domains that are different from the ones it was trained on, and propose a new approach called D3G. Unlike previous methods that aim to learn a single model that is domain invariant, D3G leverages domain similarities based on domain metadata to learn domain-specific models. Concretely, D3G learns a set of training-domain-specific functions during the training stage and reweights them based on domain relations during the test stage. These domain relations can be directly obtained and learned from domain metadata. Under mild assumptions, we theoretically prove that using domain relations to reweight training-domain-specific functions achieves stronger out-of-domain generalization compared to the conventional averaging approach. Empirically, we evaluate the effectiveness of D3G using real-world datasets for tasks such as temperature regression, land use classification, and molecule-protein binding affinity prediction. Our results show that D3G consistently outperforms state-of-the-art methods.