mlsanityData to 7 Oct 2026

Under the Surface: Tracking the Artifactuality of LLM-Generated Data

arXiv 2401.14698 ↗First posted 26 Jan 2024cs.CL cs.AIcited by 23

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

This work delves into the expanding role of large language models (LLMs) in generating artificial data. LLMs are increasingly employed to create a variety of outputs, including annotations, preferences, instruction prompts, simulated dialogues, and free text. As these forms of LLM-generated data often intersect in their application, they exert mutual influence on each other and raise significant concerns about the quality and diversity of the artificial data incorporated into training cycles, leading to an artificial data ecosystem. To the best of our knowledge, this is the first study to aggregate various types of LLM-generated text data, from more tightly constrained data like "task labels" to more lightly constrained "free-form text". We then stress test the quality and implications of LLM-generated artificial data, comparing it with human data across various existing benchmarks. Despite artificial data's capability to match human performance, this paper reveals significant hidden disparities, especially in complex tasks where LLMs often miss the nuanced understanding of intrinsic human-generated content. This study critically examines diverse LLM-generated data and emphasizes the need for ethical practices in data creation and when using LLMs. It highlights the LLMs' shortcomings in replicating human traits and behaviors, underscoring the importance of addressing biases and artifacts produced in LLM-generated content for future research and development. All data and code are available on our project page.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 55 of its 97 references that are in the map. First posted on arXiv 26 Jan 2024.

Builds on

55 of 97 references.

References · 55

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  10. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  11. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  12. 2023Large Language Models are not Fair Evaluatorscited by 1,050
38 more
  1. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  2. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  3. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  4. 2023Instruction Tuning with GPT-4cited by 676
  5. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  6. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  7. 2023How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detectioncited by 472
  8. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 414
  9. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  10. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
  11. 2023Can AI-Generated Text be Reliably Detected?cited by 372
  12. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 343
  13. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 297
  14. 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 253
  15. 2023Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaborationcited by 246
  16. 2018Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfercited by 209
  17. 2022Is GPT-3 a Good Data Annotator?cited by 208
  18. 2023ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learningcited by 203
  19. 2021Want To Reduce Labeling Cost? GPT-3 Can Helpcited by 196
  20. 2023Is ChatGPT a Good Recommender? A Preliminary Studycited by 192
  21. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  22. 2023Benchmarking Foundation Models with Language-Model-as-an-Examinercited by 172
  23. 2023Sources of Hallucination by Large Language Models on Inference Taskscited by 168
  24. 2023Benchmarking Cognitive Biases in Large Language Models as Evaluatorscited by 147
  25. 2023MAGE: Machine-generated Text Detection in the Wildcited by 130
  26. 2023Human-like Summarization Evaluation with ChatGPTcited by 126
  27. 2021On Releasing Annotator-Level Labels and Information in Datasetscited by 123
  28. 2023NLPositionality: Characterizing Design Biases of Datasets and Modelscited by 91
  29. 2023When the Majority is Wrong: Modeling Annotator Disagreement for Subjective Taskscited by 89
  30. 2023Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loopcited by 75
  31. 2020DynaSent: A Dynamic Benchmark for Sentiment Analysiscited by 63
  32. 2023Testing the Reliability of ChatGPT for Text Annotation and Classification: A Cautionary Remarkcited by 54
  33. 2023Everyone's Voice Matters: Quantifying Annotation Disagreement Using Demographic Informationcited by 52
  34. 2023ICE-Score: Instructing Large Language Models to Evaluate Codecited by 50
  35. 2023The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification Taskscited by 35
  36. 2022Crosslingual Generalization through Multitask Finetuningcited by 26
  37. 2021Does BERT Learn as Humans Perceive? Understanding Linguistic Styles through Lexicacited by 23
  38. 2023How Far Can We Extract Diverse Perspectives from Large Language Models?cited by 22

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 8; 2026: 5.

23 papers.

Sort:

By area:

  1. 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 97
  2. 2024Bias in Large Language Models: Origin, Evaluation, and Mitigationcited by 77
  3. 2024Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMscited by 59
  4. 2024Multi-modal Situated Reasoning in 3D Scenescited by 50
  5. 2024Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedbackcited by 36
  6. 2024AI and the Problem of Knowledge Collapsecited by 34
  7. 2026When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturationcited by 29
  8. 2024PolygloToxicityPrompts: Multilingual Evaluation of Neural Toxic Degeneration in Large Language Modelscited by 16
  9. 2024Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language Modelscited by 13
  10. 2025What Has Been Lost with Synthetic Evaluation?cited by 8
  11. 2024MegaFake: A Theory-Driven Dataset of Fake News Generated by Large Language Modelscited by 5
  12. 2025BharatBBQ: A Multilingual Bias Benchmark for Question Answering in the Indian Contextcited by 5
11 more
  1. 2025On the Empirical Power of Goodness-of-Fit Tests in Watermark Detectioncited by 5
  2. 2025Synthetic Artifact Auditing: Tracing LLM-Generated Synthetic Data Usage in Downstream Applicationscited by 4
  3. 2025Localizing Persona Representations in LLMscited by 3
  4. 2025How LLMs Comprehend Temporal Meaning in Narratives: A Case Study in Cognitive Evaluation of LLMscited by 3
  5. 2024Uncovering Factor Level Preferences to Improve Human-Model Alignmentcited by 2
  6. 2025Aligning Extraction and Generation for Robust Retrieval-Augmented Generationcited by 2
  7. 2026Online LLM watermark detection via e-processescited by 1
  8. 2025LLM-Generated Negative News Headlines Dataset: Creation and Benchmarking Against Real Journalismcited by 0
  9. 2026Stability as a Liability:Systematic Breakdown of Linguistic Structure in LLMscited by 0
  10. 2026Adversarial Creation and Detection of AI-Generated Social Bot Contentcited by 0
  11. 2026Benevolent Bias in Multi-Turn Human-Agent Dialoguecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data
  2. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Code
  3. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations
  4. 2024Forewarned is Forearmed: Leveraging LLMs for Data Synthesis through Failure-Inducing Exploration
  5. 2025Neither Valid nor Reliable? Investigating the Use of LLMs as Judges
  6. 2025Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls
  7. 2023Exploring the Reliability of Large Language Models as Customized Evaluators for Diverse NLP Tasks
  8. 2024Quality Matters: Evaluating Synthetic Data for Tool-Using LLMs