mlsanityData to 7 Oct 2026

Generating Synthetic Text Data to Evaluate Causal Inference Methods

arXiv 2102.05638 ↗First posted 10 Feb 2021cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

Drawing causal conclusions from observational data requires making assumptions about the true data-generating process. Causal inference research typically considers low-dimensional data, such as categorical or numerical fields in structured medical records. High-dimensional and unstructured data such as natural language complicates the evaluation of causal inference methods; such evaluations rely on synthetic datasets with known causal effects. Models for natural language generation have been widely studied and perform well empirically. However, existing methods not immediately applicable to producing synthetic datasets for causal evaluations, as they do not allow for quantifying a causal effect on the text itself. In this work, we develop a framework for adapting existing generation models to produce synthetic text datasets with known causal effects. We use this framework to perform an empirical comparison of four recently-proposed methods for estimating causal effects from text data. We release our code and synthetic datasets.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 33 of its 80 references that are in the map. First posted on arXiv 10 Feb 2021.

Builds on

33 of 80 references.

References · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  5. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  6. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  7. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  8. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  9. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  10. 2016Learning Representations for Counterfactual Inferencecited by 327
  11. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 304
  12. 2018Scalable and accurate deep learning for electronic health recordscited by 275
21 more
  1. 2017Generating Multi-label Discrete Patient Records using Generative Adversarial Networkscited by 197
  2. 2019Unifying Human and Statistical Evaluation for Natural Language Generationcited by 127
  3. 2017Automated versus do-it-yourself methods for causal inference: Lessons learned from a data analysis competitioncited by 106
  4. 2018On the Impact of Various Types of Noise on Neural Machine Translationcited by 76
  5. 2020MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Modelscited by 74
  6. 2018Deep Neural Networks for Estimation and Inferencecited by 62
  7. 2018How to Make Causal Inferences Using Textscited by 59
  8. 2020Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimatescited by 59
  9. 2019Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentencescited by 55
  10. 2020RealCause: Realistic Causal Inference Benchmarkingcited by 44
  11. 2018Benchmarking Framework for Performance-Evaluation of Causal Inference Analysiscited by 33
  12. 2018Challenges of Using Text Classifiers for Causal Inferencecited by 33
  13. 2019The Case for Evaluating Causal Models Using Interventional Measures and Empirical Datacited by 30
  14. 2018Combining multiple observational data sources to estimate causal effectscited by 29
  15. 2019Atlantic Causal Inference Conference (ACIC) Data Analysis Challenge 2017cited by 23
  16. 2018Matching with Text Data: An Experimental Evaluation of Methods for Matching Documents and of Measuring Match Qualitycited by 21
  17. 2019Towards Automatic Generation of Shareable Synthetic Clinical Notes Using Neural Language Modelscited by 20
  18. 2018Deep EHR: Chronic Disease Prediction Using Medical Notescited by 15
  19. 2020Adjusting for Confounders with Text: Challenges and an Empirical Evaluation Framework for Causal Inferencecited by 13
  20. 2018When do Words Matter? Understanding the Impact of Lexical Choice on Audience Perception using Individual Treatment Effect Estimationcited by 4
  21. 2018Towards the Creation of a Large Corpus of Synthetically-Identified Clinical Notescited by 1

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 1; 2023: 3; 2024: 3.

9 papers.

  1. 2021Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyondcited by 177
  2. 2024A Synthetic Dataset for Personal Attribute Inferencecited by 35
  3. 2021SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasetscited by 18
  4. 2022Conceptualizing Treatment Leakage in Text-based Causal Inferencecited by 14
  5. 2023RCT Rejection Sampling for Causal Estimation Evaluationcited by 7
  6. 2023Causal Inference from Text: Unveiling Interactions between Variablescited by 5
  7. 2024Mitigating Sycophancy in Decoder-Only Transformer Architectures: Synthetic Data Interventioncited by 5
  8. 2023Boosting Synthetic Data Generation with Effective Nonlinear Causal Discoverycited by 4
  9. 2024Reliability of Topic Modelingcited by 1