# Causal Information Splitting: Engineering Proxy Features for Robustness to Distribution Shifts

arXiv: https://arxiv.org/abs/2305.05832 · HTML: https://mlsanity.com/paper/2305.05832/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 43 references that are in the map.
- First posted on arXiv 10 May 2023.

## Builds on (most cited first, 15 of 15)

- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [Invariant Causal Prediction for Nonlinear Models](https://mlsanity.com/paper/1706.08576/) (2017)
- [Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions](https://mlsanity.com/paper/1707.06422/) (2017)
- [Anchor regression: heterogeneous data meets causality](https://mlsanity.com/paper/1801.06229/) (2018)
- [Regularizing towards Causal Invariance: Linear Models with Proxies](https://mlsanity.com/paper/2103.02477/) (2021)
- [Data fission: splitting a single data point](https://mlsanity.com/paper/2112.11079/) (2021)
- [Learning Counterfactually Invariant Predictors](https://mlsanity.com/paper/2207.09768/) (2022)

## Built on by (most cited first, 2 of 2)

- [Meta-Dependence in Conditional Independence Testing](https://mlsanity.com/paper/2504.12594/) (2025)
- [Data Augmentation via Causal-Residual Bootstrapping](https://mlsanity.com/paper/2603.15335/) (2026)

## Abstract (arXiv)

Statistical prediction models are often trained on data from different probability distributions than their eventual use cases. One approach to proactively prepare for these shifts harnesses the intuition that causal mechanisms should remain invariant between environments. Here we focus on a challenging setting in which the causal and anticausal variables of the target are unobserved. Leaning on information theory, we develop feature selection and engineering techniques for the observed downstream variables that act as proxies. We identify proxies that help to build stable models and moreover utilize auxiliary training tasks to answer counterfactual questions that extract stability-enhancing information from proxies. We demonstrate the effectiveness of our techniques on synthetic and real data.