# Causal Inference on Outcomes Learned from Text

arXiv: https://arxiv.org/abs/2503.00725 · HTML: https://mlsanity.com/paper/2503.00725/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 49 references that are in the map.
- First posted on arXiv 2 Mar 2025.

## Builds on (most cited first, 22 of 22)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning](https://mlsanity.com/paper/2312.01552/) (2023)
- [Prediction-Powered Inference](https://mlsanity.com/paper/2301.09633/) (2023)
- [Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond](https://mlsanity.com/paper/2109.00725/) (2021)
- [Machine Learning Methods Economists Should Know About](https://mlsanity.com/paper/1903.10075/) (2019)
- [Goal Driven Discovery of Distributional Differences via Language Descriptions](https://mlsanity.com/paper/2302.14233/) (2023)
- [How to Make Causal Inferences Using Texts](https://mlsanity.com/paper/1802.02163/) (2018)
- [Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimates](https://mlsanity.com/paper/2005.00649/) (2020)
- [Describing Differences between Text Distributions with Natural Language](https://mlsanity.com/paper/2201.12323/) (2022)
- [AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Prediction](https://mlsanity.com/paper/2305.09620/) (2023)
- [Predictions as Surrogates: Revisiting Surrogate Outcomes in the Age of AI](https://mlsanity.com/paper/2501.09731/) (2025)
- [Large Language Models: An Applied Econometric Framework](https://mlsanity.com/paper/2412.07031/) (2024)
- [Matching with Text Data: An Experimental Evaluation of Methods for Matching Documents and of Measuring Match Quality](https://mlsanity.com/paper/1801.00644/) (2018)
- [Causal Inference with Generative Artificial Intelligence: Application to Texts as Treatments](https://mlsanity.com/paper/2410.00903/) (2024)
- [GSCLIP : A Framework for Explaining Distribution Shifts in Natural Language](https://mlsanity.com/paper/2206.15007/) (2022)
- [CAREER: A Foundation Model for Labor Sequence Data](https://mlsanity.com/paper/2202.08370/) (2022)
- [ElicitationGPT: Text Elicitation Mechanisms via Language Models](https://mlsanity.com/paper/2406.09363/) (2024)
- [Measuring the Completeness of Theories](https://mlsanity.com/paper/1910.07022/) (2019)
- [LABOR-LLM: Language-Based Occupational Representations with Large Language Models](https://mlsanity.com/paper/2406.17972/) (2024)
- [Program Evaluation with Remotely Sensed Outcomes](https://mlsanity.com/paper/2411.10959/) (2024)
- [Machine-Learning Tests for Effects on Multiple Outcomes](https://mlsanity.com/paper/1707.01473/) (2017)

## Built on by (most cited first, 8 of 8)

- [This human study did not involve human subjects: Validating LLM simulations as behavioral evidence](https://mlsanity.com/paper/2602.15785/) (2026)
- [ComplLLM: Fine-tuning LLMs to Discover Complementary Signals for Decision-making](https://mlsanity.com/paper/2602.19458/) (2026)
- [Advertising in AI systems: Society must be vigilant](https://mlsanity.com/paper/2505.18425/) (2025)
- [E-LDA: Toward Interpretable LDA Topic Models with Strong Guarantees in Logarithmic Parallel Time](https://mlsanity.com/paper/2506.07747/) (2025)
- [Causal Effect Estimation with Latent Textual Treatments](https://mlsanity.com/paper/2602.15730/) (2026)
- [Conditional Hypothesis Generation for LLM-Based Text Analysis with Researcher-Specified Covariates](https://mlsanity.com/paper/2606.03029/) (2026)
- [Econometrics with Pre-Trained Embeddings for Unstructured Data](https://mlsanity.com/paper/2607.17378/) (2026)
- [Causal Inference with Unstructured Outcomes](https://mlsanity.com/paper/2608.03085/) (2026)

## Abstract (arXiv)

We propose a machine-learning tool that yields causal inference on text in randomized trials. Based on a simple econometric framework in which text may capture outcomes of interest, our procedure addresses three questions: First, is the text affected by the treatment? Second, which outcomes is the effect on? And third, how complete is our description of causal effects? To answer all three questions, our approach uses large language models (LLMs) that suggest systematic differences across two groups of text documents and then provides valid inference based on costly validation. Specifically, we highlight the need for sample splitting to allow for statistical validation of LLM outputs, as well as the need for human labeling to validate substantive claims about how documents differ across groups. We illustrate the tool in a proof-of-concept application using abstracts of academic manuscripts.