# Invariant and Transportable Representations for Anti-Causal Domain Shifts

arXiv: https://arxiv.org/abs/2207.01603 · HTML: https://mlsanity.com/paper/2207.01603/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 56 references that are in the map.
- First posted on arXiv 4 Jul 2022.

## Builds on (most cited first, 30 of 36)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Learning to reinforcement learn](https://mlsanity.com/paper/1611.05763/) (2016)
- [Gradient Matching for Domain Generalization](https://mlsanity.com/paper/2104.09937/) (2021)
- [Weakly-Supervised Disentanglement Without Compromises](https://mlsanity.com/paper/2002.02886/) (2020)
- [Few-Shot Learning via Learning the Representation, Provably](https://mlsanity.com/paper/2002.09434/) (2020)
- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [Representation Learning via Invariant Causal Mechanisms](https://mlsanity.com/paper/2010.07922/) (2020)
- [Provable Meta-Learning of Linear Representations](https://mlsanity.com/paper/2002.11684/) (2020)
- [Invariant Causal Prediction for Nonlinear Models](https://mlsanity.com/paper/1706.08576/) (2017)
- [Causality Inspired Representation Learning for Domain Generalization](https://mlsanity.com/paper/2203.14237/) (2022)
- [On Calibration and Out-of-domain Generalization](https://mlsanity.com/paper/2102.10395/) (2021)
- [Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions](https://mlsanity.com/paper/1707.06422/) (2017)
- [Anchor regression: heterogeneous data meets causality](https://mlsanity.com/paper/1801.06229/) (2018)
- [DIVA: Domain Invariant Variational Autoencoders](https://mlsanity.com/paper/1905.10427/) (2019)
- [Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests](https://mlsanity.com/paper/2106.00545/) (2021)
- [Desiderata for Representation Learning: A Causal Perspective](https://mlsanity.com/paper/2109.03795/) (2021)
- [Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalization](https://mlsanity.com/paper/2202.06856/) (2022)
- [Learning Causal Semantic Representation for Out-of-Distribution Prediction](https://mlsanity.com/paper/2011.01681/) (2020)
- [Counterfactuals uncover the modular structure of deep generative models](https://mlsanity.com/paper/1812.03253/) (2018)

## Built on by (most cited first, 30 of 37)

- [On Finding Small Hyper-Gradients in Bilevel Optimization: Hardness Results and Improved Analysis](https://mlsanity.com/paper/2301.00712/) (2023)
- [Mind the Label Shift of Augmentation-based Graph OOD Generalization](https://mlsanity.com/paper/2303.14859/) (2023)
- [Leveraging sparse and shared feature activations for disentangled representation learning](https://mlsanity.com/paper/2304.07939/) (2023)
- [Unbiased Faster R-CNN for Single-source Domain Generalized Object Detection](https://mlsanity.com/paper/2405.15225/) (2024)
- [Spuriosity Didn't Kill the Classifier: Using Invariant Predictions to Harness Spurious Features](https://mlsanity.com/paper/2307.09933/) (2023)
- [Efficient Conditionally Invariant Representation Learning](https://mlsanity.com/paper/2212.08645/) (2022)
- [Data Augmentations for Improved (Large) Language Model Generalization](https://mlsanity.com/paper/2310.12803/) (2023)
- [MIST: Defending Against Membership Inference Attacks Through Membership-Invariant Subspace Training](https://mlsanity.com/paper/2311.00919/) (2023)
- [No Fair Lunch: A Causal Perspective on Dataset Bias in Machine Learning for Medical Imaging](https://mlsanity.com/paper/2307.16526/) (2023)
- [Do causal predictors generalize better to new domains?](https://mlsanity.com/paper/2402.09891/) (2024)
- [Causal Inference via Style Transfer for Out-of-distribution Generalisation](https://mlsanity.com/paper/2212.03063/) (2022)
- [Invariant Anomaly Detection under Distribution Shifts: A Causal Perspective](https://mlsanity.com/paper/2312.14329/) (2023)
- [Rethinking Fair Representation Learning for Performance-Sensitive Tasks](https://mlsanity.com/paper/2410.04120/) (2024)
- [Beyond Invariance: Test-Time Label-Shift Adaptation for Distributions with "Spurious" Correlations](https://mlsanity.com/paper/2211.15646/) (2022)
- [Interpreting CLIP: Insights on the Robustness to ImageNet Distribution Shifts](https://mlsanity.com/paper/2310.13040/) (2023)
- [Prominent Roles of Conditionally Invariant Components in Domain Adaptation: Theory and Algorithms](https://mlsanity.com/paper/2309.10301/) (2023)
- [Out-of-Variable Generalization for Discriminative Models](https://mlsanity.com/paper/2304.07896/) (2023)
- [START: A Generalized State Space Model with Saliency-Driven Token-Aware Transformation](https://mlsanity.com/paper/2410.16020/) (2024)
- [Measure-Theoretic Anti-Causal Representation Learning](https://mlsanity.com/paper/2510.18052/) (2025)
- [SSL Framework for Causal Inconsistency between Structures and Representations](https://mlsanity.com/paper/2310.18634/) (2023)
- [Differentially Private Bilevel Optimization](https://mlsanity.com/paper/2409.19800/) (2024)
- [Deconfounding Causal Inference through Two-Branch Framework with Early-Forking for Sensor-Based Cross-Domain Activity Recognition](https://mlsanity.com/paper/2507.03898/) (2025)
- [Learning Invariant Representations with a Nonparametric Nadaraya-Watson Head](https://mlsanity.com/paper/2309.13377/) (2023)
- [Diagnosing and Rectifying Fake OOD Invariance: A Restructured Causal Approach](https://mlsanity.com/paper/2312.09758/) (2023)
- [Multi-environment Topic Models](https://mlsanity.com/paper/2410.24126/) (2024)
- [CPCANet: Deep Unfolding Common Principal Component Analysis for Domain Generalization](https://mlsanity.com/paper/2605.05136/) (2026)
- [Structure-agnostic Causal Representation Learning](https://mlsanity.com/paper/2610.00968/) (2026)
- [Towards Causal Representation Learning and Deconfounding from Indefinite Data](https://mlsanity.com/paper/2305.02640/) (2023)
- [Accelerated Fully First-Order Methods for Bilevel and Minimax Optimization](https://mlsanity.com/paper/2405.00914/) (2024)
- [Causal Fine-Tuning under Latent Confounded Shift](https://mlsanity.com/paper/2410.14375/) (2024)

## Abstract (arXiv)

Real-world classification problems must contend with domain shift, the (potential) mismatch between the domain where a model is deployed and the domain(s) where the training data was gathered. Methods to handle such problems must specify what structure is common between the domains and what varies. A natural assumption is that causal (structural) relationships are invariant in all domains. Then, it is tempting to learn a predictor for label Y that depends only on its causal parents. However, many real-world problems are "anti-causal" in the sense that Y is a cause of the covariates X -- in this case, Y has no causal parents and the naive causal invariance is useless. In this paper, we study representation learning under a particular notion of domain shift that both respects causal invariance and that naturally handles the "anti-causal" structure. We show how to leverage the shared causal structure of the domains to learn a representation that both admits an invariant predictor and that also allows fast adaptation in new domains. The key is to translate causal assumptions into learning principles that disentangle "invariant" and "non-stable" features. Experiments on both synthetic and real-world data demonstrate the effectiveness of the proposed learning algorithm. Code is available at https://github.com/ybjiaang/ACTIR.