# Exploiting Domain-Specific Features to Enhance Domain Generalization

arXiv: https://arxiv.org/abs/2110.09410 · HTML: https://mlsanity.com/paper/2110.09410/

## Key facts

- Cited by 86 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 18 Oct 2021.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Self-Challenging Improves Cross-Domain Generalization](https://mlsanity.com/paper/2007.02454/) (2020)
- [Domain Generalization for Object Recognition with Multi-task Autoencoders](https://mlsanity.com/paper/1508.07680/) (2015)
- [Adversarial Domain Adaptation with Domain Mixup](https://mlsanity.com/paper/1912.01805/) (2019)
- [Emergence of Invariance and Disentanglement in Deep Representations](https://mlsanity.com/paper/1706.01350/) (2017)
- [Learning Robust Representations via Multi-View Information Bottleneck](https://mlsanity.com/paper/2002.07017/) (2020)
- [Improve Unsupervised Domain Adaptation with Mixup Training](https://mlsanity.com/paper/2001.00677/) (2020)
- [Domain Generalization by Marginal Transfer Learning](https://mlsanity.com/paper/1711.07910/) (2017)
- [Learning to Optimize Domain Specific Normalization for Domain Generalization](https://mlsanity.com/paper/1907.04275/) (2019)
- [Domain Generalization via Conditional Invariant Representation](https://mlsanity.com/paper/1807.08479/) (2018)
- [Efficient Domain Generalization via Common-Specific Low-Rank Decomposition](https://mlsanity.com/paper/2003.12815/) (2020)
- [Learning to Balance Specificity and Invariance for In and Out of Domain Generalization](https://mlsanity.com/paper/2008.12839/) (2020)
- [DIVA: Domain Invariant Variational Autoencoders](https://mlsanity.com/paper/1905.10427/) (2019)
- [Heterogeneous Domain Generalization via Domain Mixup](https://mlsanity.com/paper/2009.05448/) (2020)
- [Support and Invertibility in Domain-Invariant Representations](https://mlsanity.com/paper/1903.03448/) (2019)
- [Domain Generalization via Multidomain Discriminant Analysis](https://mlsanity.com/paper/1907.11216/) (2019)
- [Estimating Generalization under Distribution Shifts via Domain-Invariant Representations](https://mlsanity.com/paper/2007.03511/) (2020)
- [On Learning Invariant Representation for Domain Adaptation](https://mlsanity.com/paper/1901.09453/) (2019)

## Built on by (most cited first, 30 of 86)

- [Domain Generalization by Mutual-Information Regularization with Pre-trained Models](https://mlsanity.com/paper/2203.10789/) (2022)
- [Sharpness-Aware Gradient Matching for Domain Generalization](https://mlsanity.com/paper/2303.10353/) (2023)
- [Sparse Mixture-of-Experts are Domain Generalizable Learners](https://mlsanity.com/paper/2206.04046/) (2022)
- [Leveraging Vision-Language Models for Improving Domain Generalization in Image Classification](https://mlsanity.com/paper/2310.08255/) (2023)
- [Self-Distilled Vision Transformer for Domain Generalization](https://mlsanity.com/paper/2207.12392/) (2022)
- [Domain-invariant Feature Exploration for Domain Generalization](https://mlsanity.com/paper/2207.12020/) (2022)
- [Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectives](https://mlsanity.com/paper/2307.16851/) (2023)
- [Domain Generalization by Learning and Removing Domain-specific Features](https://mlsanity.com/paper/2212.07101/) (2022)
- [MADG: Margin-based Adversarial Learning for Domain Generalization](https://mlsanity.com/paper/2311.08503/) (2023)
- [Domain-Unified Prompt Representations for Source-Free Domain Generalization](https://mlsanity.com/paper/2209.14926/) (2022)
- [Spuriosity Didn't Kill the Classifier: Using Invariant Predictions to Harness Spurious Features](https://mlsanity.com/paper/2307.09933/) (2023)
- [DART: Diversify-Aggregate-Repeat Training Improves Generalization of Neural Networks](https://mlsanity.com/paper/2302.14685/) (2023)
- [ERM++: An Improved Baseline for Domain Generalization](https://mlsanity.com/paper/2304.01973/) (2023)
- [Improving Domain Generalization with Domain Relations](https://mlsanity.com/paper/2302.02609/) (2023)
- [Energy-Based Test Sample Adaptation for Domain Generalization](https://mlsanity.com/paper/2302.11215/) (2023)
- [Density-Softmax: Efficient Test-time Model for Uncertainty Estimation and Robustness under Distribution Shifts](https://mlsanity.com/paper/2302.06495/) (2023)
- [Domain Generalization via Nuclear Norm Regularization](https://mlsanity.com/paper/2303.07527/) (2023)
- [V2X-DGW: Domain Generalization for Multi-agent Perception under Adverse Weather Conditions](https://mlsanity.com/paper/2403.11371/) (2024)
- [Controlling Neural Collapse Enhances Out-of-Distribution Detection and Transfer Learning](https://mlsanity.com/paper/2502.10691/) (2025)
- [Decoupled Self-supervised Learning for Non-Homophilous Graphs](https://mlsanity.com/paper/2206.03601/) (2022)
- [Rethinking Generalization: The Impact of Annotation Style on Medical Image Segmentation](https://mlsanity.com/paper/2210.17398/) (2022)
- [Domain Generalization with Correlated Style Uncertainty](https://mlsanity.com/paper/2212.09950/) (2022)
- [TFS-ViT: Token-Level Feature Stylization for Domain Generalization](https://mlsanity.com/paper/2303.15698/) (2023)
- [Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes](https://mlsanity.com/paper/2412.13573/) (2024)
- [Adaptive Domain Generalization via Online Disagreement Minimization](https://mlsanity.com/paper/2208.01996/) (2022)
- [Gradient Estimation for Unseen Domain Risk Minimization with Pre-Trained Models](https://mlsanity.com/paper/2302.01497/) (2023)
- [HPL-ESS: Hybrid Pseudo-Labeling for Unsupervised Event-based Semantic Segmentation](https://mlsanity.com/paper/2403.16788/) (2024)
- [Bridging Domain Generalization to Multimodal Domain Generalization via Unified Representations](https://mlsanity.com/paper/2507.03304/) (2025)
- [Aggregation of Disentanglement: Reconsidering Domain Variations in Domain Generalization](https://mlsanity.com/paper/2302.02350/) (2023)
- [NormAUG: Normalization-guided Augmentation for Domain Generalization](https://mlsanity.com/paper/2307.13492/) (2023)

## Abstract (arXiv)

Domain Generalization (DG) aims to train a model, from multiple observed source domains, in order to perform well on unseen target domains. To obtain the generalization capability, prior DG approaches have focused on extracting domain-invariant information across sources to generalize on target domains, while useful domain-specific information which strongly correlates with labels in individual domains and the generalization to target domains is usually ignored. In this paper, we propose meta-Domain Specific-Domain Invariant (mDSDI) - a novel theoretically sound framework that extends beyond the invariance view to further capture the usefulness of domain-specific information. Our key insight is to disentangle features in the latent space while jointly learning both domain-invariant and domain-specific features in a unified framework. The domain-specific representation is optimized through the meta-learning framework to adapt from source domains, targeting a robust generalization on unseen domains. We empirically show that mDSDI provides competitive results with state-of-the-art techniques in DG. A further ablation study with our generated dataset, Background-Colored-MNIST, confirms the hypothesis that domain-specific is essential, leading to better results when compared with only using domain-invariant.