# Efficient Domain Generalization via Common-Specific Low-Rank Decomposition

arXiv: https://arxiv.org/abs/2003.12815 · HTML: https://mlsanity.com/paper/2003.12815/

## Key facts

- Cited by 134 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Mar 2020.

## Built on by (most cited first, 30 of 134)

- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [A Fourier-based Framework for Domain Generalization](https://mlsanity.com/paper/2105.11120/) (2021)
- [Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.06607/) (2021)
- [Deep Stable Learning for Out-Of-Distribution Generalization](https://mlsanity.com/paper/2104.07876/) (2021)
- [Causality Inspired Representation Learning for Domain Generalization](https://mlsanity.com/paper/2203.14237/) (2022)
- [Uncertainty Modeling for Out-of-Distribution Generalization](https://mlsanity.com/paper/2202.03958/) (2022)
- [Open Domain Generalization with Domain-Augmented Meta-Learning](https://mlsanity.com/paper/2104.03620/) (2021)
- [Model-Based Domain Generalization](https://mlsanity.com/paper/2102.11436/) (2021)
- [CLIPood: Generalizing CLIP to Out-of-Distributions](https://mlsanity.com/paper/2302.00864/) (2023)
- [Exploiting Domain-Specific Features to Enhance Domain Generalization](https://mlsanity.com/paper/2110.09410/) (2021)
- [Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2108.11249/) (2021)
- [Domain Invariant Representation Learning with Domain Density Transformations](https://mlsanity.com/paper/2102.05082/) (2021)
- [Non-Transferable Learning: A New Approach for Model Ownership Verification and Applicability Authorization](https://mlsanity.com/paper/2106.06916/) (2021)
- [SimMMDG: A Simple and Effective Framework for Multi-modal Domain Generalization](https://mlsanity.com/paper/2310.19795/) (2023)
- [Meta-causal Learning for Single Domain Generalization](https://mlsanity.com/paper/2304.03709/) (2023)
- [Leveraging Vision-Language Models for Improving Domain Generalization in Image Classification](https://mlsanity.com/paper/2310.08255/) (2023)
- [Towards Unsupervised Domain Generalization](https://mlsanity.com/paper/2107.06219/) (2021)
- [Compound Domain Generalization via Meta-Knowledge Encoding](https://mlsanity.com/paper/2203.13006/) (2022)
- [Federated Domain Generalization for Image Recognition via Cross-Client Style Transfer](https://mlsanity.com/paper/2210.00912/) (2022)
- [Domain Generalization via Inference-time Label-Preserving Target Projections](https://mlsanity.com/paper/2103.01134/) (2021)
- [Anomaly Detection under Distribution Shift](https://mlsanity.com/paper/2303.13845/) (2023)
- [Modeling the Data-Generating Process is Necessary for Out-of-Distribution Generalization](https://mlsanity.com/paper/2206.07837/) (2022)
- [Domain-invariant Feature Exploration for Domain Generalization](https://mlsanity.com/paper/2207.12020/) (2022)
- [An Empirical Framework for Domain Generalization in Clinical Settings](https://mlsanity.com/paper/2103.11163/) (2021)
- [A Style and Semantic Memory Mechanism for Domain Generalization](https://mlsanity.com/paper/2112.07517/) (2021)
- [Neuron Structure Modeling for Generalizable Remote Physiological Measurement](https://mlsanity.com/paper/2303.05955/) (2023)
- [On the Out-Of-Distribution Generalization of Multimodal Large Language Models](https://mlsanity.com/paper/2402.06599/) (2024)
- [BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning](https://mlsanity.com/paper/2203.01522/) (2022)
- [Training for the Future: A Simple Gradient Interpolation Loss to Generalize Along Time](https://mlsanity.com/paper/2108.06721/) (2021)

## Abstract (arXiv)

Domain generalization refers to the task of training a model which generalizes to new domains that are not seen during training. We present CSD (Common Specific Decomposition), for this setting,which jointly learns a common component (which generalizes to new domains) and a domain specific component (which overfits on training domains). The domain specific components are discarded after training and only the common component is retained. The algorithm is extremely simple and involves only modifying the final linear classification layer of any given neural network architecture. We present a principled analysis to understand existing approaches, provide identifiability results of CSD,and study effect of low-rank on domain generalization. We show that CSD either matches or beats state of the art approaches for domain generalization based on domain erasure, domain perturbed data augmentation, and meta-learning. Further diagnostics on rotated MNIST, where domains are interpretable, confirm the hypothesis that CSD successfully disentangles common and domain specific components and hence leads to better domain generalization.