mlsanityData to 7 Oct 2026

DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows

arXiv 2402.10379 ↗First posted 16 Feb 2024cs.CL cs.LGcited by 43

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

Large language models (LLMs) have become a dominant and important tool for NLP researchers in a wide range of tasks. Today, many researchers use LLMs in synthetic data generation, task evaluation, fine-tuning, distillation, and other model-in-the-loop research workflows. However, challenges arise when using these models that stem from their scale, their closed source nature, and the lack of standardized tooling for these new and emerging workflows. The rapid rise to prominence of these models and these unique challenges has had immediate adverse impacts on open science and on the reproducibility of work that uses them. In this paper, we introduce DataDreamer, an open source Python library that allows researchers to write simple code to implement powerful LLM workflows. DataDreamer also helps researchers adhere to best practices that we propose to encourage open science and reproducibility. The library and documentation are available at https://github.com/datadreamer-dev/DataDreamer .

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 35 of its 49 references that are in the map. First posted on arXiv 16 Feb 2024.

Builds on

35 of 49 references.

References · 35

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  10. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  11. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  12. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
23 more
  1. 2020Learning to summarize from human feedbackcited by 2,562
  2. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  3. 2018Model Cards for Model Reportingcited by 1,068
  4. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  5. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  6. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  7. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  8. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  9. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  10. 2022Large Language Models Can Self-Improvecited by 656
  11. 2024TinyLlama: An Open-Source Small Language Modelcited by 604
  12. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  13. 2021Datasets: A Community Library for Natural Language Processingcited by 526
  14. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  15. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 369
  16. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 343
  17. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 247
  18. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 230
  19. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 209
  20. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 185
  21. 2022Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AIcited by 169
  22. 2023Benchmarking and Improving Generator-Validator Consistency of Language Modelscited by 43
  23. 2021Unsupervised Neural Machine Translation with Generative Language Models Onlycited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 19; 2026: 10.

43 papers.

Sort:

By area:

  1. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Codecited by 61
  2. 2024Large Language Models Can Self-Improve At Web Agent Taskscited by 54
  3. 2025Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generationcited by 54
  4. 2025Language Self-Play For Data-Free Trainingcited by 42
  5. 2024AI-Assisted Generation of Difficult Math Questionscited by 37
  6. 2024SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuningcited by 32
  7. 2024StyleDistance: Stronger Content-Independent Style Embeddings with Synthetic Parallel Examplescited by 32
  8. 2024Is Programming by Example solved by LLMs?cited by 27
  9. 2025Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfallscited by 24
  10. 2024Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspectivecited by 15
  11. 2024MIND: Math Informed syNthetic Dialogues for Pretraining LLMscited by 14
  12. 2026A Survey on LLM-based Conversational User Simulationcited by 14
31 more
  1. 2024On Pre-training of Multimodal Language Models Customized for Chart Understandingcited by 13
  2. 2024LLM for Barcodes: Generating Diverse Synthetic Data for Identity Documentscited by 12
  3. 2025LLMs can be easily Confused by Instructional Distractionscited by 12
  4. 2025Beyond Translation: LLM-Based Data Generation for Multilingual Fact-Checkingcited by 8
  5. 2025SynthTextEval: Synthetic Text Data Generation and Evaluation for High-Stakes Domainscited by 7
  6. 2025Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learningcited by 7
  7. 2026On Protecting Agentic Systems' Intellectual Property via Watermarkingcited by 7
  8. 2024Exploring Empty Spaces: Human-in-the-Loop Data Augmentationcited by 6
  9. 2024SS-GEN: A Social Story Generation Framework with Large Language Modelscited by 4
  10. 2024Improving Text Generation on Images with Synthetic Captionscited by 3
  11. 2024Training Task Experts through Retrieval Based Distillationcited by 3
  12. 2025Polypersona: Persona-Grounded LLM for Synthetic Survey Responsescited by 3
  13. 2025Chart-CoCa: Self-Improving Chart Understanding of Vision LMs via Code-Driven Synthesis and Candidate-Conditioned Answeringcited by 2
  14. 2025mStyleDistance: Multilingual Style Embeddings and their Evaluationcited by 1
  15. 2025High-Fidelity And Complex Test Data Generation For Google SQL Code Generation Servicescited by 1
  16. 2025An LLM-Empowered Low-Resolution Vision System for On-Device Human Behavior Understandingcited by 1
  17. 2025Multi-Model Synthetic Training for Mission-Critical Small Language Modelscited by 1
  18. 2025Annotating Training Data for Conditional Semantic Textual Similarity Measurement using Large Language Modelscited by 1
  19. 2025LLM Unlearning on Noisy Forget Sets: A Study of Incomplete, Rewritten, and Watermarked Datacited by 1
  20. 2025Structured Prompting and LLM Ensembling for Multimodal Conversational Aspect-based Sentiment Analysiscited by 1
  21. 2026Team, Then Trim: An Assembly-Line LLM Framework for High-Quality Tabular Data Generationcited by 1
  22. 2024Did You Hear That? Introducing AADG: A Framework for Generating Benchmark Data in Audio Anomaly Detectioncited by 0
  23. 2025Unlocking the Potential of Large Language Models in the Nuclear Industry with Synthetic Datacited by 0
  24. 2025Language Models Might Not Understand You: Evaluating Theory of Mind via Story Promptingcited by 0
  25. 2026The Critical Role of Aspects in Measuring Document Similaritycited by 0
  26. 2026FineInstructions: Scaling Synthetic Instructions to Pre-Training Scalecited by 0
  27. 2026Temporal Flattening in LLM-Generated Text: Comparing Human and LLM Writing Trajectoriescited by 0
  28. 2026Does Synthetic Data Help? Empirical Evidence from Deep Learning Time Series Forecasterscited by 0
  29. 2026Context-Instrumental Data Distillation for Kubernetes Manifest Generation: Method and Experimental Evaluationcited by 0
  30. 2026Synthetic Worlds for Temporal Evaluation and Knowledge Updating in LLMscited by 0
  31. 2026Overlap, Unique and Conflict: Can LLMs Extract What They Can Recognize?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data
  2. 2025LMR-BENCH: Evaluating LLM Agent's Ability on Reproducing Language Modeling Research
  3. 2026Better as Generators Than Classifiers: Leveraging LLMs and Synthetic Data for Low-Resource Multilingual Classification
  4. 2024A Survey on Data Synthesis and Augmentation for Large Language Models
  5. 2024Efficacy of Synthetic Data as a Benchmark
  6. 2025AIRepr: An Analyst-Inspector Framework for Evaluating Reproducibility of LLMs in Data Science
  7. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey
  8. 2025BARE: Leveraging Base Language Models for Few-Shot Synthetic Data Generation