# Causal Inference with Generative Artificial Intelligence: Application to Texts as Treatments

arXiv: https://arxiv.org/abs/2410.00903 · HTML: https://mlsanity.com/paper/2410.00903/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 61 references that are in the map.
- First posted on arXiv 1 Oct 2024.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [Text and Code Embeddings by Contrastive Pre-Training](https://mlsanity.com/paper/2201.10005/) (2022)
- [Regularisation of Neural Networks by Enforcing Lipschitz Continuity](https://mlsanity.com/paper/1804.04368/) (2018)
- [Adapting Neural Networks for the Estimation of Treatment Effects](https://mlsanity.com/paper/1906.02120/) (2019)
- [Fine-Tuning LLaMA for Multi-Stage Text Retrieval](https://mlsanity.com/paper/2310.08319/) (2023)
- [A Contrastive Framework for Neural Text Generation](https://mlsanity.com/paper/2202.06417/) (2022)
- [Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond](https://mlsanity.com/paper/2109.00725/) (2021)
- [The Blessings of Multiple Causes](https://mlsanity.com/paper/1805.06826/) (2018)
- [Scaling Sentence Embeddings with Large Language Models](https://mlsanity.com/paper/2307.16645/) (2023)
- [Desiderata for Representation Learning: A Causal Perspective](https://mlsanity.com/paper/2109.03795/) (2021)
- [How to Make Causal Inferences Using Texts](https://mlsanity.com/paper/1802.02163/) (2018)
- [Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimates](https://mlsanity.com/paper/2005.00649/) (2020)
- [A Causal Lens for Controllable Text Generation](https://mlsanity.com/paper/2201.09119/) (2022)
- [Causal Effects of Linguistic Properties](https://mlsanity.com/paper/2010.12919/) (2020)
- [Matching with Text Data: An Experimental Evaluation of Methods for Matching Documents and of Measuring Match Quality](https://mlsanity.com/paper/1801.00644/) (2018)
- [Image-based Treatment Effect Heterogeneity](https://mlsanity.com/paper/2206.06417/) (2022)
- [Causal Estimation for Text Data with (Apparent) Overlap Violations](https://mlsanity.com/paper/2210.00079/) (2022)
- [Conceptualizing Treatment Leakage in Text-based Causal Inference](https://mlsanity.com/paper/2205.00465/) (2022)
- [DoubleMLDeep: Estimation of Causal Effects with Multimodal Data](https://mlsanity.com/paper/2402.01785/) (2024)
- [Integrating Earth Observation Data into Causal Inference: Challenges and Opportunities](https://mlsanity.com/paper/2301.12985/) (2023)
- [Leveraging text data for causal inference using electronic health records](https://mlsanity.com/paper/2307.03687/) (2023)

## Built on by (most cited first, 16 of 16)

- [This human study did not involve human subjects: Validating LLM simulations as behavioral evidence](https://mlsanity.com/paper/2602.15785/) (2026)
- [Causal Inference on Outcomes Learned from Text](https://mlsanity.com/paper/2503.00725/) (2025)
- [A Design-based Solution for Causal Inference with Text: Can a Language Model Be Too Large?](https://mlsanity.com/paper/2510.08758/) (2025)
- [Isolated Causal Effects of Natural Language](https://mlsanity.com/paper/2410.14812/) (2024)
- [Leveraging Generative Artificial Intelligence for Causal Inference with Unstructured Data](https://mlsanity.com/paper/2507.03897/) (2025)
- [Static Sandboxes Are Inadequate: Modeling Societal Complexity Requires Open-Ended Co-Evolution in LLM-Based Multi-Agent Simulations](https://mlsanity.com/paper/2510.13982/) (2025)
- [Leveraging semantic similarity for experimentation with AI-generated treatments](https://mlsanity.com/paper/2510.21119/) (2025)
- [Generator-Mediated Bandits: Thompson Sampling for GenAI-Powered Adaptive Interventions](https://mlsanity.com/paper/2505.16311/) (2025)
- [GraphRAG-Causal: A novel graph-augmented framework for causal reasoning and annotation in news](https://mlsanity.com/paper/2506.11600/) (2025)
- [Policy Learning with a Natural Language Action Space: A Causal Approach](https://mlsanity.com/paper/2502.17538/) (2025)
- [Mitigating Hidden Confounding by Progressive Confounder Imputation via Large Language Models](https://mlsanity.com/paper/2507.02928/) (2025)
- [Causal Effect Estimation with Latent Textual Treatments](https://mlsanity.com/paper/2602.15730/) (2026)
- [Supervised Learning Objectives for High-Dimensional Treatment Effects](https://mlsanity.com/paper/2605.27281/) (2026)
- [Predicting Causal Effects from Natural Language Queries using Structured Representations](https://mlsanity.com/paper/2605.29631/) (2026)
- [Vector Search As Nearest Neighbor Matching: RAG-based Policy Learning in Causal Inference](https://mlsanity.com/paper/2607.18225/) (2026)
- [Research Design Tracking and Assessment for the Social Sciences](https://mlsanity.com/paper/2608.27049/) (2026)

## Abstract (arXiv)

In this paper, we demonstrate how to enhance the validity of causal inference with unstructured high-dimensional treatments like texts, by leveraging the power of generative Artificial Intelligence (GenAI). Specifically, we propose to use a deep generative model such as large language models (LLMs) to efficiently generate treatments and use their internal representation for subsequent causal effect estimation. We show that the knowledge of this true internal representation helps disentangle the treatment features of interest, such as specific sentiments and certain topics, from other possibly unknown confounding features. Unlike existing methods, the proposed GenAI-Powered Inference (GPI) methodology eliminates the need to learn causal representation from the data, and hence produces more accurate and efficient estimates. We formally establish the conditions required for the nonparametric identification of the average treatment effect, propose an estimation strategy that avoids the violation of the overlap assumption, and derive the asymptotic properties of the proposed estimator through the application of double machine learning. Finally, using an instrumental variables approach, we extend the proposed GPI methodology to the settings in which the treatment feature is based on human perception. The GPI is also applicable to text reuse where an LLM is used to regenerate existing texts. We conduct simulation and empirical studies, using the generated text data from an open-source LLM, Llama 3, to illustrate the advantages of our estimator over state-of-the-art causal representation learning algorithms.