# Learning Infomax and Domain-Independent Representations for Causal Effect Inference with Real-World Data

arXiv: https://arxiv.org/abs/2202.10885 · HTML: https://mlsanity.com/paper/2202.10885/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 31 references that are in the map.
- First posted on arXiv 22 Feb 2022.

## Builds on (most cited first, 10 of 10)

- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Learning Representations for Counterfactual Inference](https://mlsanity.com/paper/1605.03661/) (2016)
- [A Survey on Causal Inference](https://mlsanity.com/paper/2002.02770/) (2020)
- [Bayesian Inference of Individualized Treatment Effects using Multi-task Gaussian Processes](https://mlsanity.com/paper/1704.02801/) (2017)
- [Learning Counterfactual Representations for Estimating Individual Dose-Response Curves](https://mlsanity.com/paper/1902.00981/) (2019)
- [Perfect Match: A Simple Method for Learning Representations For Counterfactual Inference With Neural Networks](https://mlsanity.com/paper/1810.00656/) (2018)
- [Graph Infomax Adversarial Learning for Treatment Effect Estimation with Networked Observational Data](https://mlsanity.com/paper/2106.02881/) (2021)
- [Matching in Selective and Balanced Representation Space for Treatment Effects Estimation](https://mlsanity.com/paper/2009.06828/) (2020)

## Built on by (most cited first, 9 of 9)

- [A Causal Explainable Guardrails for Large Language Models](https://mlsanity.com/paper/2405.04160/) (2024)
- [Causal Effect Estimation: Recent Advances, Challenges, and Opportunities](https://mlsanity.com/paper/2302.00848/) (2023)
- [Continual Causal Inference with Incremental Observational Data](https://mlsanity.com/paper/2303.01775/) (2023)
- [A Survey of Deep Causal Models and Their Industrial Applications](https://mlsanity.com/paper/2209.08860/) (2022)
- [Causal Contrastive Learning for Counterfactual Regression Over Time](https://mlsanity.com/paper/2406.00535/) (2024)
- [Multi-Task Adversarial Learning for Treatment Effect Estimation in Basket Trials](https://mlsanity.com/paper/2203.05123/) (2022)
- [Hierarchical Capsule Prediction Network for Marketing Campaigns Effect](https://mlsanity.com/paper/2208.10113/) (2022)
- [Incorporating Causal Analysis into Diversified and Logical Response Generation](https://mlsanity.com/paper/2209.09482/) (2022)
- [Continual Causal Effect Estimation: Challenges and Opportunities](https://mlsanity.com/paper/2301.01026/) (2023)

## Abstract (arXiv)

The foremost challenge to causal inference with real-world data is to handle the imbalance in the covariates with respect to different treatment options, caused by treatment selection bias. To address this issue, recent literature has explored domain-invariant representation learning based on different domain divergence metrics (e.g., Wasserstein distance, maximum mean discrepancy, position-dependent metric, and domain overlap). In this paper, we reveal the weaknesses of these strategies, i.e., they lead to the loss of predictive information when enforcing the domain invariance; and the treatment effect estimation performance is unstable, which heavily relies on the characteristics of the domain distributions and the choice of domain divergence metrics. Motivated by information theory, we propose to learn the Infomax and Domain-Independent Representations to solve the above puzzles. Our method utilizes the mutual information between the global feature representations and individual feature representations, and the mutual information between feature representations and treatment assignment predictions, in order to maximally capture the common predictive information for both treatment and control groups. Moreover, our method filters out the influence of instrumental and irrelevant variables, and thus it effectively increases the predictive ability of potential outcomes. Experimental results on both the synthetic and real-world datasets show that our method achieves state-of-the-art performance on causal effect inference. Moreover, our method exhibits reliable prediction performances when facing data with different characteristics of data distributions, complicated variable types, and severe covariate imbalance.