mlsanityData to 7 Oct 2026

Synthetically generated text for supervised text analysis

arXiv 2303.16028 ↗First posted 28 Mar 2023cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

Supervised text models are a valuable tool for political scientists but present several obstacles to their use, including the expense of hand-labeling documents, the difficulty of retrieving rare relevant documents for annotation, and copyright and privacy concerns involved in sharing annotated documents. This article proposes a partial solution to these three issues, in the form of controlled generation of synthetic text with large language models. I provide a conceptual overview of text generation, guidance on when researchers should prefer different techniques for generating synthetic text, a discussion of ethics, and a simple technique for improving the quality of synthetic text. I demonstrate the usefulness of synthetic text with three applications: generating synthetic tweets describing the fighting in Ukraine, synthetic news articles describing specified political events for training an event detection system, and a multilingual corpus of populist manifesto statements for training a sentence-level populism classifier.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 25 of its 80 references that are in the map. First posted on arXiv 28 Mar 2023.

Builds on

25 of 80 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  5. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  6. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  7. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  8. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  9. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  10. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  11. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  12. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
13 more
  1. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  2. 2019Defending Against Neural Fake Newscited by 553
  3. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 377
  4. 2019Automatic Detection of Generated Text is Easiest when Humans are Fooledcited by 260
  5. 2021Generating Datasets with Pretrained Language Modelscited by 192
  6. 2022Efficient Few-Shot Learning Without Promptscited by 135
  7. 2018Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysiscited by 110
  8. 2018Tensor2Tensor for Neural Machine Translationcited by 85
  9. 2020A Theoretical Analysis of the Repetition Problem in Text Generationcited by 83
  10. 2021Adaptive Semiparametric Language Modelscited by 83
  11. 2020Exploring Controllable Text Generation Techniquescited by 55
  12. 2021Generate, Annotate, and Learn: NLP with Synthetic Textcited by 42
  13. 2021Corpus-Level Evaluation for Event QA: The IndiaPoliceEvents Corpus Covering the 2002 Gujarat Violencecited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 1; 2025: 3; 2026: 2.

10 papers.