mlsanityData to 14 Oct 2026

Language Models as Causal Effect Generators

arXiv 2411.08019 ↗First posted 12 Nov 2024cs.CL cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

In this work, we present sequence-driven structural causal models (SD-SCMs), a framework for specifying causal models with user-defined structure and language-model-defined mechanisms. We characterize how an SD-SCM enables sampling from observational, interventional, and counterfactual distributions according to the desired causal structure. We then leverage this procedure to propose a new type of benchmark for causal inference methods, generating individual-level counterfactual data to test treatment effect estimation. We create an example benchmark consisting of thousands of datasets, and test a suite of popular estimation methods for average, conditional average, and individual treatment effect estimation. We find under this benchmark that (1) causal methods outperform non-causal methods and that (2) even state-of-the-art methods struggle with individualized effect estimation, suggesting this benchmark captures some inherent difficulties in causal estimation. Apart from generating data, this same technique can underpin the auditing of language models for (un)desirable causal effects, such as misinformation or discrimination. We believe SD-SCMs can serve as a useful tool in any application that would benefit from sequential data with controllable causal structure.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 28 of its 68 references that are in the map. First posted on arXiv 12 Nov 2024.

Builds on

28 of 68 references.

References · 28

  1. 2017Counterfactual Fairnesscited by 804
  2. 2015Estimation and Inference of Heterogeneous Treatment Effects using Random Forestscited by 600
  3. 2017Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learningcited by 478
  4. 2017Causal Effect Inference with Deep Latent-Variable Modelscited by 410
  5. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 353
  6. 2017Quasi-Oracle Estimation of Heterogeneous Treatment Effectscited by 332
  7. 2016Generalized Random Forestscited by 312
  8. 2020Deep Structural Causal Models for Tractable Counterfactual Inferencecited by 211
  9. 2018Path-Specific Counterfactual Fairnesscited by 181
  10. 2021Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyondcited by 178
  11. 2020Conformal Inference of Counterfactuals and Individual Treatment Effectscited by 151
  12. 2023Causal Parrots: Large Language Models May Talk Causality But Are Not Causalcited by 150
16 more
  1. 2023Can Large Language Models Infer Causation from Correlation?cited by 146
  2. 2021Nonparametric Estimation of Heterogeneous Treatment Effects: From Theory to Learning Algorithmscited by 144
  3. 2022Diffusion Causal Models for Counterfactual Estimationcited by 87
  4. 2020RealCause: Realistic Causal Inference Benchmarkingcited by 46
  5. 2023Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactualscited by 44
  6. 2023Prompting Large Language Models for Counterfactual Generation: An Empirical Studycited by 37
  7. 2019The Case for Evaluating Causal Models Using Interventional Measures and Empirical Datacited by 30
  8. 2017Orthogonal Machine Learning: Power and Limitationscited by 28
  9. 2021CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoningcited by 23
  10. 2024A Survey on Natural Language Counterfactual Generationcited by 21
  11. 2020How and Why to Use Experimental Data to Evaluate Methods for Observational Causal Inferencecited by 13
  12. 2022Flexible text generation for counterfactual fairness probingcited by 13
  13. 2017Synth-Validation: Selecting the Best Causal Inference Method for a Given Datasetcited by 9
  14. 2021Sketch and Customize: A Counterfactual Story Generatorcited by 9
  15. 2021On the Distinction Between "Conditional Average Treatment Effects" (CATE) and "Individual Treatment Effects" (ITE) Under Ignorability Assumptionscited by 9
  16. 2024Deep Autoregressive Models as Causal Inference Enginescited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Black Box Causal Inference: Effect Estimation via Meta Predictioncited by 13
  2. 2025PROPHET: An Inferable Future Forecasting Benchmark with Causal Intervened Likelihood Estimationcited by 8
  3. 2026Linear-LLM-SCM: Benchmarking LLMs for Coefficient Elicitation in Linear-Gaussian Causal Modelscited by 1
  4. 2026CausalDS: Benchmarking Causal Reasoning in Data-Science Agentscited by 1