# End-To-End Causal Effect Estimation from Unstructured Natural Language Data

arXiv: https://arxiv.org/abs/2407.07018 · HTML: https://mlsanity.com/paper/2407.07018/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 72 references that are in the map.
- First posted on arXiv 9 Jul 2024.

## Builds on (most cited first, 27 of 27)

- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [Towards a Human-like Open-Domain Chatbot](https://mlsanity.com/paper/2001.09977/) (2020)
- [Causal Reasoning and Large Language Models: Opening a New Frontier for Causality](https://mlsanity.com/paper/2305.00050/) (2023)
- [The Pushshift Reddit Dataset](https://mlsanity.com/paper/2001.08435/) (2020)
- [Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond](https://mlsanity.com/paper/2109.00725/) (2021)
- [Causal Parrots: Large Language Models May Talk Causality But Are Not Causal](https://mlsanity.com/paper/2308.13067/) (2023)
- [Can Large Language Models Infer Causation from Correlation?](https://mlsanity.com/paper/2306.05836/) (2023)
- [Can large language models build causal graphs?](https://mlsanity.com/paper/2303.05279/) (2023)
- [How to Make Causal Inferences Using Texts](https://mlsanity.com/paper/1802.02163/) (2018)
- [Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimates](https://mlsanity.com/paper/2005.00649/) (2020)
- [Integrating Large Language Model for Improved Causal Discovery](https://mlsanity.com/paper/2306.16902/) (2023)
- [Efficient Causal Graph Discovery Using Large Language Models](https://mlsanity.com/paper/2402.01207/) (2024)
- [Causal Order: The Key to Leveraging Imperfect Experts in Causal Inference](https://mlsanity.com/paper/2310.15117/) (2023)
- [Identification and estimation of treatment and interference effects in observational studies on networks](https://mlsanity.com/paper/1609.06245/) (2016)
- [Causal Effects of Linguistic Properties](https://mlsanity.com/paper/2010.12919/) (2020)
- [Causal-Discovery Performance of ChatGPT in the context of Neuropathic Pain Diagnosis](https://mlsanity.com/paper/2301.13819/) (2023)
- [Personalized Decision Making -- A Conceptual Introduction](https://mlsanity.com/paper/2208.09558/) (2022)
- [Estimating Causal Effects of Tone in Online Debates](https://mlsanity.com/paper/1906.04177/) (2019)
- [Zero-shot Causal Graph Extrapolation from Text via LLMs](https://mlsanity.com/paper/2312.14670/) (2023)
- [A First Course in Causal Inference](https://mlsanity.com/paper/2305.18793/) (2023)
- [Applying Large Language Models for Causal Structure Learning in Non Small Cell Lung Cancer](https://mlsanity.com/paper/2311.07191/) (2023)
- [RCT Rejection Sampling for Causal Estimation Evaluation](https://mlsanity.com/paper/2307.15176/) (2023)
- [Causal Lifting and Link Prediction](https://mlsanity.com/paper/2302.01198/) (2023)
- [Causal Inference With Outcome-Dependent Missingness And Self-Censoring](https://mlsanity.com/paper/2306.05511/) (2023)
- [Bayesian Topic Regression for Causal Inference](https://mlsanity.com/paper/2109.05317/) (2021)
- [Flexible sensitivity analysis for causal inference in observational studies subject to unmeasured confounding](https://mlsanity.com/paper/2305.17643/) (2023)

## Built on by (most cited first, 18 of 18)

- [Causality for Large Language Models](https://mlsanity.com/paper/2410.15319/) (2024)
- [OCEAN: Offline Chain-of-thought Evaluation and Alignment in Large Language Models](https://mlsanity.com/paper/2410.23703/) (2024)
- [Efficient Randomized Experiments Using Foundation Models](https://mlsanity.com/paper/2502.04262/) (2025)
- [A Survey on Enhancing Causal Reasoning Ability of Large Language Models](https://mlsanity.com/paper/2503.09326/) (2025)
- [Counterfactual Causal Inference in Natural Language with Large Language Models](https://mlsanity.com/paper/2410.06392/) (2024)
- [Ice Cream Doesn't Cause Drowning: Benchmarking LLMs Against Statistical Pitfalls in Causal Inference](https://mlsanity.com/paper/2505.13770/) (2025)
- [Adjustment for Confounding using Pre-Trained Representations](https://mlsanity.com/paper/2506.14329/) (2025)
- [Causal Inference with Complex Treatments: A Survey](https://mlsanity.com/paper/2407.14022/) (2024)
- [CausalEvolve: Towards Open-Ended Discovery with Causal Scratchpad](https://mlsanity.com/paper/2603.14575/) (2026)
- [Isolated Causal Effects of Natural Language](https://mlsanity.com/paper/2410.14812/) (2024)
- [LLM-Driven Treatment Effect Estimation Under Inference Time Text Confounding](https://mlsanity.com/paper/2507.02843/) (2025)
- [ReLU integral probability metric and its applications](https://mlsanity.com/paper/2504.18897/) (2025)
- [Mitigating Hidden Confounding by Progressive Confounder Imputation via Large Language Models](https://mlsanity.com/paper/2507.02928/) (2025)
- [LLM-based Agents for Automated Confounder Discovery and Subgroup Analysis in Causal Inference](https://mlsanity.com/paper/2508.07221/) (2025)
- [Bayesian Sensitivity of Causal Inference Estimators under Evidence-Based Priors](https://mlsanity.com/paper/2605.07993/) (2026)
- [Why Prompt Optimization Works, and Why It Sometimes Doesn't: A Causal-Inspired Edit-Level Analysis](https://mlsanity.com/paper/2605.26655/) (2026)
- [Supervised Learning Objectives for High-Dimensional Treatment Effects](https://mlsanity.com/paper/2605.27281/) (2026)
- [Uncertainty-Guided LLM Semantic Augmentation for Heterogeneous Treatment Effect Estimation](https://mlsanity.com/paper/2607.26599/) (2026)

## Abstract (arXiv)

Knowing the effect of an intervention is critical for human decision-making, but current approaches for causal effect estimation rely on manual data collection and structuring, regardless of the causal assumptions. This increases both the cost and time-to-completion for studies. We show how large, diverse observational text data can be mined with large language models (LLMs) to produce inexpensive causal effect estimates under appropriate causal assumptions. We introduce NATURAL, a novel family of causal effect estimators built with LLMs that operate over datasets of unstructured text. Our estimators use LLM conditional distributions (over variables of interest, given the text data) to assist in the computation of classical estimators of causal effect. We overcome a number of technical challenges to realize this idea, such as automating data curation and using LLMs to impute missing information. We prepare six (two synthetic and four real) observational datasets, paired with corresponding ground truth in the form of randomized trials, which we used to systematically evaluate each step of our pipeline. NATURAL estimators demonstrate remarkable performance, yielding causal effect estimates that fall within 3 percentage points of their ground truth counterparts, including on real-world Phase 3/4 clinical trials. Our results suggest that unstructured text data is a rich source of causal effect information, and NATURAL is a first step towards an automated pipeline to tap this resource.