# Isolated Causal Effects of Natural Language

arXiv: https://arxiv.org/abs/2410.14812 · HTML: https://mlsanity.com/paper/2410.14812/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 44 references that are in the map.
- First posted on arXiv 18 Oct 2024.

## Builds on (most cited first, 21 of 21)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [What is the Effect of Importance Weighting in Deep Learning?](https://mlsanity.com/paper/1812.03372/) (2018)
- ["Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters](https://mlsanity.com/paper/2310.09219/) (2023)
- [Semiparametric doubly robust targeted double machine learning: a review](https://mlsanity.com/paper/2203.06469/) (2022)
- [Fairness And Bias in Artificial Intelligence: A Brief Survey of Sources, Impacts, And Mitigation Strategies](https://mlsanity.com/paper/2304.07683/) (2023)
- [How to Make Causal Inferences Using Texts](https://mlsanity.com/paper/1802.02163/) (2018)
- [Empath: Understanding Topic Signals in Large-Scale Text](https://mlsanity.com/paper/1602.06979/) (2016)
- [Causal Effects of Linguistic Properties](https://mlsanity.com/paper/2010.12919/) (2020)
- [Doubly Robust Distributionally Robust Off-Policy Evaluation and Learning](https://mlsanity.com/paper/2202.09667/) (2022)
- [Long Story Short: Omitted Variable Bias in Causal Machine Learning](https://mlsanity.com/paper/2112.13398/) (2021)
- [End-To-End Causal Effect Estimation from Unstructured Natural Language Data](https://mlsanity.com/paper/2407.07018/) (2024)
- [Causal Inference with Generative Artificial Intelligence: Application to Texts as Treatments](https://mlsanity.com/paper/2410.00903/) (2024)
- [Optimizing Language Models for Human Preferences is a Causal Inference Problem](https://mlsanity.com/paper/2402.14979/) (2024)
- [Text-Transport: Toward Learning Causal Effects of Natural Language](https://mlsanity.com/paper/2310.20697/) (2023)
- [SenteCon: Leveraging Lexicons to Learn Human-Interpretable Language Representations](https://mlsanity.com/paper/2305.14728/) (2023)
- [Towards Representation Learning for Weighting Problems in Design-Based Causal Inference](https://mlsanity.com/paper/2409.16407/) (2024)

## Built on by (most cited first, 2 of 2)

- [Omitted Variable Bias in Language Models Under Distribution Shift](https://mlsanity.com/paper/2602.16784/) (2026)
- [Social Circuits behind Multi-agent Echo Chambers](https://mlsanity.com/paper/2609.34444/) (2026)

## Abstract (arXiv)

As language technologies become widespread, it is important to understand how changes in language affect reader perceptions and behaviors. These relationships may be formalized as the isolated causal effect of some focal language-encoded intervention (e.g., factual inaccuracies) on an external outcome (e.g., readers' beliefs). In this paper, we introduce a formal estimation framework for isolated causal effects of language. We show that a core challenge of estimating isolated effects is the need to approximate all non-focal language outside of the intervention. Drawing on the principle of omitted variable bias, we provide measures for evaluating the quality of both non-focal language approximations and isolated effect estimates themselves. We find that poor approximation of non-focal language can lead to bias in the corresponding isolated effect estimates due to omission of relevant variables, and we show how to assess the sensitivity of effect estimates to such bias along the two key axes of fidelity and overlap. In experiments on semi-synthetic and real-world data, we validate the ability of our framework to correctly recover isolated effects and demonstrate the utility of our proposed measures.