# Text Rationalization for Robust Causal Effect Estimation

arXiv: https://arxiv.org/abs/2512.05373 · HTML: https://mlsanity.com/paper/2512.05373/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 64 references that are in the map.
- First posted on arXiv 5 Dec 2025.

## Builds on (most cited first, 30 of 35)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [Causal Effect Inference with Deep Latent-Variable Models](https://mlsanity.com/paper/1705.08821/) (2017)
- [Adapting Neural Networks for the Estimation of Treatment Effects](https://mlsanity.com/paper/1906.02120/) (2019)
- [Rationalizing Neural Predictions](https://mlsanity.com/paper/1606.04155/) (2016)
- [Causal Discovery with Continuous Additive Noise Models](https://mlsanity.com/paper/1309.6779/) (2013)
- [Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond](https://mlsanity.com/paper/2109.00725/) (2021)
- [Interpretable Neural Predictions with Differentiable Binary Variables](https://mlsanity.com/paper/1905.08160/) (2019)
- [Learning to Faithfully Rationalize by Construction](https://mlsanity.com/paper/2005.00115/) (2020)
- [Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control](https://mlsanity.com/paper/1910.13294/) (2019)
- [Probabilities of Causation: Bounds and Identification](https://mlsanity.com/paper/1301.3898/) (2013)
- [Approximate Kernel-based Conditional Independence Tests for Fast Non-Parametric Causal Discovery](https://mlsanity.com/paper/1702.03877/) (2017)
- [Balancing Covariates via Propensity Score Weighting](https://mlsanity.com/paper/1609.07494/) (2016)
- [Treatment effect estimation with disentangled latent factors](https://mlsanity.com/paper/2001.10652/) (2020)
- [How to Make Causal Inferences Using Texts](https://mlsanity.com/paper/1802.02163/) (2018)
- [Deriving Machine Attention from Human Rationales](https://mlsanity.com/paper/1808.09367/) (2018)
- [Challenges of Using Text Classifiers for Causal Inference](https://mlsanity.com/paper/1810.00956/) (2018)
- [On the Testability of Causal Models with Latent and Instrumental Variables](https://mlsanity.com/paper/1302.4976/) (2013)
- [Learning from the Best: Rationalizing Prediction by Adversarial Information Calibration](https://mlsanity.com/paper/2012.08884/) (2020)
- [Self-Compatibility: Evaluating Causal Discovery without Ground Truth](https://mlsanity.com/paper/2307.09552/) (2023)
- [How Does Selective Mechanism Improve Self-Attention Networks?](https://mlsanity.com/paper/2005.00979/) (2020)
- [Rationalization through Concepts](https://mlsanity.com/paper/2105.04837/) (2021)
- [SPECTRA: Sparse Structured Text Rationalization](https://mlsanity.com/paper/2109.04552/) (2021)
- [Towards Trustworthy Explanation: On Causal Rationalization](https://mlsanity.com/paper/2306.14115/) (2023)

## Built on by (most cited first, 2 of 2)

- [Causal Inference with Unstructured Treatments](https://mlsanity.com/paper/2608.00657/) (2026)
- [FinInvest-GTCN: Explainable Graph-Temporal-Causal Modeling for Risk-Aware Investment Decision Optimization](https://mlsanity.com/paper/2606.28933/) (2026)

## Abstract (arXiv)

Recent advances in natural language processing have enabled the increasing use of text data in causal inference, particularly for adjusting confounding factors in treatment effect estimation. Although high-dimensional text can encode rich contextual information, it also poses unique challenges for causal identification and estimation. In particular, the positivity assumption, which requires sufficient treatment overlap across confounder values, is often violated at the observational level, when massive text is represented in feature spaces. Redundant or spurious textual features inflate dimensionality, producing extreme propensity scores, unstable weights, and inflated variance in effect estimates. We address these challenges with Confounding-Aware Token Rationalization (CATR), a framework that selects a sparse necessary subset of tokens using a residual-independence diagnostic designed to preserve confounding information sufficient for unconfoundedness. By discarding irrelevant texts while retaining key signals, CATR mitigates observational-level positivity violations and stabilizes downstream causal effect estimators. Experiments on synthetic data and a real-world study using the MIMIC-III database demonstrate that CATR yields more accurate, stable, and interpretable causal effect estimates than existing baselines.