Invariant Feature Extraction Through Conditional Independence and the Optimal Transport Barycenter Problem: the Gaussian case

A methodology is developed to extract d invariant features W=f(X) that predict a response variable Y without being confounded by variables Z that may influence both X and Y. The methodology's main ingredient is the penalization of any statistical dependence between W and Z conditioned on Y, replaced by the more readily implementable plain independence between W and the random variable ZY = T(Z,Y) that solves the [Monge] Optimal Transport Barycenter Problem for Zmid Y. In the Gaussian case considered in this article, the two statements are equivalent. When the true confounders Z are unknown, other measurable contextual variables S can be used as surrogates, a replacement that involves no relaxation in the Gaussian case if the covariance matrix SigmaZS has full range. The resulting linear feature extractor adopts a closed form in terms of the first d eigenvectors of a known matrix. The procedure extends with little change to more general, non-Gaussian / non-linear cases.