mlsanityData to 13 Oct 2026

Causal Inference on Outcomes Learned from Text

arXiv 2503.00725 ↗First posted 2 Mar 2025econ.EM cs.CL cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

We propose a machine-learning tool that yields causal inference on text in randomized trials. Based on a simple econometric framework in which text may capture outcomes of interest, our procedure addresses three questions: First, is the text affected by the treatment? Second, which outcomes is the effect on? And third, how complete is our description of causal effects? To answer all three questions, our approach uses large language models (LLMs) that suggest systematic differences across two groups of text documents and then provides valid inference based on costly validation. Specifically, we highlight the need for sample splitting to allow for statistical validation of LLM outputs, as well as the need for human labeling to validate substantive claims about how documents differ across groups. We illustrate the tool in a proof-of-concept application using abstracts of academic manuscripts.

Key facts. Cited by 8 papers in the map (data through 13 Oct 2026). Builds on 22 of its 49 references that are in the map. First posted on arXiv 2 Mar 2025.

Builds on

22 of 49 references.

References · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,320
  2. 2023LIMA: Less Is More for Alignmentcited by 1,091
  3. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 255
  4. 2023Prediction-Powered Inferencecited by 218
  5. 2021Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyondcited by 178
  6. 2019Machine Learning Methods Economists Should Know Aboutcited by 64
  7. 2023Goal Driven Discovery of Distributional Differences via Language Descriptionscited by 61
  8. 2018How to Make Causal Inferences Using Textscited by 59
  9. 2020Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimatescited by 59
  10. 2022Describing Differences between Text Distributions with Natural Languagecited by 54
  11. 2023AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Predictioncited by 47
  12. 2025Predictions as Surrogates: Revisiting Surrogate Outcomes in the Age of AIcited by 32
10 more
  1. 2024Large Language Models: An Applied Econometric Frameworkcited by 31
  2. 2018Matching with Text Data: An Experimental Evaluation of Methods for Matching Documents and of Measuring Match Qualitycited by 21
  3. 2024Causal Inference with Generative Artificial Intelligence: Application to Texts as Treatmentscited by 16
  4. 2022GSCLIP : A Framework for Explaining Distribution Shifts in Natural Languagecited by 14
  5. 2022CAREER: A Foundation Model for Labor Sequence Datacited by 11
  6. 2024ElicitationGPT: Text Elicitation Mechanisms via Language Modelscited by 9
  7. 2019Measuring the Completeness of Theoriescited by 6
  8. 2024LABOR-LLM: Language-Based Occupational Representations with Large Language Modelscited by 6
  9. 2024Program Evaluation with Remotely Sensed Outcomescited by 3
  10. 2017Machine-Learning Tests for Effects on Multiple Outcomescited by 2

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2026This human study did not involve human subjects: Validating LLM simulations as behavioral evidencecited by 28
  2. 2025Advertising in AI systems: Society must be vigilantcited by 1
  3. 2025E-LDA: Toward Interpretable LDA Topic Models with Strong Guarantees in Logarithmic Parallel Timecited by 1
  4. 2026ComplLLM: Fine-tuning LLMs to Discover Complementary Signals for Decision-makingcited by 1
  5. 2026Causal Effect Estimation with Latent Textual Treatmentscited by 0
  6. 2026Conditional Hypothesis Generation for LLM-Based Text Analysis with Researcher-Specified Covariatescited by 0
  7. 2026Econometrics with Pre-Trained Embeddings for Unstructured Datacited by 0
  8. 2026Causal Inference with Unstructured Outcomescited by 0