# The Confounder Trap: Treatment-Encoding Representations in Causal Inference with Text

arXiv: https://arxiv.org/abs/2607.26309 · HTML: https://mlsanity.com/paper/2607.26309/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 28 Jul 2026.

## Builds on (most cited first, 14 of 14)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [LEACE: Perfect linear concept erasure in closed form](https://mlsanity.com/paper/2306.03819/) (2023)
- [How to Make Causal Inferences Using Texts](https://mlsanity.com/paper/1802.02163/) (2018)
- [Causal Effects of Linguistic Properties](https://mlsanity.com/paper/2010.12919/) (2020)
- [Matching with Text Data: An Experimental Evaluation of Methods for Matching Documents and of Measuring Match Quality](https://mlsanity.com/paper/1801.00644/) (2018)
- [Estimating Causal Effects of Tone in Online Debates](https://mlsanity.com/paper/1906.04177/) (2019)
- [Causal Estimation for Text Data with (Apparent) Overlap Violations](https://mlsanity.com/paper/2210.00079/) (2022)
- [Leveraging text data for causal inference using electronic health records](https://mlsanity.com/paper/2307.03687/) (2023)
- [Bayesian Topic Regression for Causal Inference](https://mlsanity.com/paper/2109.05317/) (2021)
- [Choosing the Number of Topics in LDA Models -- A Monte Carlo Comparison of Selection Criteria](https://mlsanity.com/paper/2212.14074/) (2022)
- [A Design-based Solution for Causal Inference with Text: Can a Language Model Be Too Large?](https://mlsanity.com/paper/2510.08758/) (2025)
- [Using Text-Based Causal Inference to Disentangle Factors Influencing Online Review Ratings](https://mlsanity.com/paper/2606.04286/) (2026)

## Abstract (arXiv)

Estimating causal effects of linguistic properties from observational text is difficult because the same document can contain both the treatment of interest and the non-treatment textual attributes needed for adjustment. Existing approaches often learn representations from the full text to capture latent confounding, but when treatment status is itself encoded by words in the text, these representations can directly encode treatment. This creates a confounder trap: richer representations can make treated and control documents separable, inducing overlap violations even when the underlying causal problem satisfies overlap. We study latent text treatments that are encoded through lexicons or other treatment-defining lexical information, and propose masking-based adjustment representations that remove this lexical treatment signal before representation learning. We formalize representation-induced overlap failure, prove that deletion masking preserves overlap for bag-of-words/topic-model representations, and characterize replacement masking as a natural relaxation for large language models that hides treatment-defining tokens while preserving word order and context. Across simulations, masking improves overlap diagnostics, stabilizes treatment effect estimates, and reduces bias relative to adjustment methods that learn from the unmasked text.