# Regularizing towards Causal Invariance: Linear Models with Proxies

arXiv: https://arxiv.org/abs/2103.02477 · HTML: https://mlsanity.com/paper/2103.02477/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 3 Mar 2021.

## Builds on (most cited first, 15 of 15)

- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [The Risks of Invariant Risk Minimization](https://mlsanity.com/paper/2010.05761/) (2020)
- [Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions](https://mlsanity.com/paper/1707.06422/) (2017)
- [Anchor regression: heterogeneous data meets causality](https://mlsanity.com/paper/1801.06229/) (2018)
- [Does Invariant Risk Minimization Capture Invariance?](https://mlsanity.com/paper/2101.01134/) (2021)
- [An Introduction to Proximal Causal Learning](https://mlsanity.com/paper/2009.10982/) (2020)
- [Robustness to Spurious Correlations via Human Annotations](https://mlsanity.com/paper/2007.06661/) (2020)
- [Distributionally Robust Losses for Latent Covariate Mixtures](https://mlsanity.com/paper/2007.13982/) (2020)
- [Out-of-distribution Prediction with Invariant Risk Minimization: The Limitation and An Effective Fix](https://mlsanity.com/paper/2101.07732/) (2021)
- [Accounting for Unobserved Confounding in Domain Generalization](https://mlsanity.com/paper/2007.10653/) (2020)
- [Distributional robustness of K-class estimators and the PULSE](https://mlsanity.com/paper/2005.03353/) (2020)
- [Distributional Anchor Regression](https://mlsanity.com/paper/2101.08224/) (2021)
- [Deconfounding and Causal Regularization for Stability and External Validity](https://mlsanity.com/paper/2008.06234/) (2020)

## Built on by (most cited first, 20 of 20)

- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Invariant Learning via Probability of Sufficient and Necessary Causes](https://mlsanity.com/paper/2309.12559/) (2023)
- [Image-based Treatment Effect Heterogeneity](https://mlsanity.com/paper/2206.06417/) (2022)
- [Invariant Policy Learning: A Causal Perspective](https://mlsanity.com/paper/2106.00808/) (2021)
- [CausPref: Causal Preference Learning for Out-of-Distribution Recommendation](https://mlsanity.com/paper/2202.03984/) (2022)
- [Learning from a Biased Sample](https://mlsanity.com/paper/2209.01754/) (2022)
- [An Empirical Study on Distribution Shift Robustness From the Perspective of Pre-Training and Data Augmentation](https://mlsanity.com/paper/2205.12753/) (2022)
- [Evaluating Robustness to Dataset Shift via Parametric Robustness Sets](https://mlsanity.com/paper/2205.15947/) (2022)
- [Proxy Methods for Domain Adaptation](https://mlsanity.com/paper/2403.07442/) (2024)
- [Beyond Discriminant Patterns: On the Robustness of Decision Rule Ensembles](https://mlsanity.com/paper/2109.10432/) (2021)
- [Exploiting Personalized Invariance for Better Out-of-distribution Generalization in Federated Learning](https://mlsanity.com/paper/2211.11243/) (2022)
- [Causal Information Splitting: Engineering Proxy Features for Robustness to Distribution Shifts](https://mlsanity.com/paper/2305.05832/) (2023)
- [Federated Adapter on Foundation Models: An Out-Of-Distribution Approach](https://mlsanity.com/paper/2505.01075/) (2025)
- [An Analysis of Causal Effect Estimation using Outcome Invariant Data Augmentation](https://mlsanity.com/paper/2510.25128/) (2025)
- [Revisiting Spurious Correlation in Domain Generalization](https://mlsanity.com/paper/2406.11517/) (2024)
- [Invariant Causal Routing for Governing Social Norms in Online Market Economies](https://mlsanity.com/paper/2603.04534/) (2026)
- [Causal Domain Adaptation with Copula Entropy based Conditional Independence Test](https://mlsanity.com/paper/2202.13482/) (2022)
- [Towards Robust Off-Policy Evaluation via Human Inputs](https://mlsanity.com/paper/2209.08682/) (2022)
- [Nonlinear Causal Discovery via Kernel Anchor Regression](https://mlsanity.com/paper/2210.16775/) (2022)
- [Out-of-distribution robustness for multivariate analysis via causal regularisation](https://mlsanity.com/paper/2403.01865/) (2024)

## Abstract (arXiv)

We propose a method for learning linear models whose predictive performance is robust to causal interventions on unobserved variables, when noisy proxies of those variables are available. Our approach takes the form of a regularization term that trades off between in-distribution performance and robustness to interventions. Under the assumption of a linear structural causal model, we show that a single proxy can be used to create estimators that are prediction optimal under interventions of bounded strength. This strength depends on the magnitude of the measurement noise in the proxy, which is, in general, not identifiable. In the case of two proxy variables, we propose a modified estimator that is prediction optimal under interventions up to a known strength. We further show how to extend these estimators to scenarios where additional information about the "test time" intervention is available during training. We evaluate our theoretical findings in synthetic experiments and using real data of hourly pollution levels across several cities in China.