Domain Generalization via Conditional Invariant Representation

Domain generalization aims to apply knowledge gained from multiple labeled source domains to unseen target domains. The main difficulty comes from the dataset bias: training data and test data have different distributions, and the training set contains heterogeneous samples from different distributions. Let X denote the features, and Y be the class labels. Existing domain generalization methods address the dataset bias problem by learning a domain-invariant representation h(X) that has the same marginal distribution mathbbP(h(X)) across multiple source domains. The functional relationship encoded in mathbbP(Y|X) is usually assumed to be stable across domains such that mathbbP(Y|h(X)) is also invariant. However, it is unclear whether this assumption holds in practical problems. In this paper, we consider the general situation where both mathbbP(X) and mathbbP(Y|X) can change across all domains. We propose to learn a feature representation which has domain-invariant class conditional distributions mathbbP(h(X)|Y). With the conditional invariant representation, the invariance of the joint distribution mathbbP(h(X),Y) can be guaranteed if the class prior mathbbP(Y) does not change across training and test domains. Extensive experiments on both synthetic and real data demonstrate the effectiveness of the proposed method.