# Learning Counterfactually Invariant Predictors

arXiv: https://arxiv.org/abs/2207.09768 · HTML: https://mlsanity.com/paper/2207.09768/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 85 references that are in the map.
- First posted on arXiv 20 Jul 2022.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [Avoiding Discrimination through Causal Reasoning](https://mlsanity.com/paper/1706.02744/) (2017)
- [Path-Specific Counterfactual Fairness](https://mlsanity.com/paper/1802.08139/) (2018)
- [Fair Inference On Outcomes](https://mlsanity.com/paper/1705.10378/) (2017)
- [Invariant Causal Prediction for Nonlinear Models](https://mlsanity.com/paper/1706.08576/) (2017)
- [Invariance, Causality and Robustness](https://mlsanity.com/paper/1812.08233/) (2018)
- [A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings](https://mlsanity.com/paper/2002.03689/) (2020)
- [Random Fourier Features for Kernel Ridge Regression: Approximation Bounds and Statistical Guarantees](https://mlsanity.com/paper/1804.09893/) (2018)
- [Causally motivated Shortcut Removal Using Auxiliary Labels](https://mlsanity.com/paper/2105.06422/) (2021)
- [Causal Reasoning for Algorithmic Fairness](https://mlsanity.com/paper/1805.05859/) (2018)
- [Characteristic and Universal Tensor Product Kernels](https://mlsanity.com/paper/1708.08157/) (2017)
- [Kernel Distribution Embeddings: Universal Kernels, Characteristic Kernels and Kernel Metrics on Distributions](https://mlsanity.com/paper/1604.05251/) (2016)
- [Metrizing Weak Convergence with Maximum Mean Discrepancies](https://mlsanity.com/paper/2006.09268/) (2020)
- [Causal Feature Selection for Algorithmic Fairness](https://mlsanity.com/paper/2006.06053/) (2020)
- [Achieving non-discrimination in data release](https://mlsanity.com/paper/1611.07438/) (2016)
- [A Unifying Causal Framework for Analyzing Dataset Shift-stable Learning Algorithms](https://mlsanity.com/paper/1905.11374/) (2019)
- [Selection, Ignorability and Challenges With Causal Fairness](https://mlsanity.com/paper/2202.13774/) (2022)
- [Fairness Under Feature Exemptions: Counterfactual and Observational Measures](https://mlsanity.com/paper/2006.07986/) (2020)
- [Algorithmic Decision Making with Conditional Fairness](https://mlsanity.com/paper/2006.10483/) (2020)

## Built on by (most cited first, 10 of 10)

- [Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment](https://mlsanity.com/paper/2501.09620/) (2025)
- [Efficient Conditionally Invariant Representation Learning](https://mlsanity.com/paper/2212.08645/) (2022)
- [When Shift Happens - Confounding Is to Blame](https://mlsanity.com/paper/2505.21422/) (2025)
- [Results on Counterfactual Invariance](https://mlsanity.com/paper/2307.08519/) (2023)
- [Prediction without Preclusion: Recourse Verification with Reachable Sets](https://mlsanity.com/paper/2308.12820/) (2023)
- [Causal Information Splitting: Engineering Proxy Features for Robustness to Distribution Shifts](https://mlsanity.com/paper/2305.05832/) (2023)
- [AI Alignment in Medical Imaging: Unveiling Hidden Biases Through Counterfactual Analysis](https://mlsanity.com/paper/2504.19621/) (2025)
- [Learning Conditional Invariances through Non-Commutativity](https://mlsanity.com/paper/2402.11682/) (2024)
- [DISCO: Mitigating Bias in Deep Learning with Conditional Distance Correlation](https://mlsanity.com/paper/2506.11653/) (2025)
- [A pipeline for enabling path-specific causal fairness in observational health data](https://mlsanity.com/paper/2601.09841/) (2026)

## Abstract (arXiv)

Notions of counterfactual invariance (CI) have proven essential for predictors that are fair, robust, and generalizable in the real world. We propose graphical criteria that yield a sufficient condition for a predictor to be counterfactually invariant in terms of a conditional independence in the observational distribution. In order to learn such predictors, we propose a model-agnostic framework, called Counterfactually Invariant Prediction (CIP), building on the Hilbert-Schmidt Conditional Independence Criterion (HSCIC), a kernel-based conditional dependence measure. Our experimental results demonstrate the effectiveness of CIP in enforcing counterfactual invariance across various simulated and real-world datasets including scalar and multi-variate settings.