mlsanityData to 7 Oct 2026

Synthetic Eggs in Many Baskets: The Impact of Synthetic Data Diversity on LLM Fine-Tuning

arXiv 2511.01490 ↗First posted 3 Nov 2025cs.CLcited by 1

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

As synthetic data becomes widely used in language model development, understanding its impact on model behavior is crucial. This paper investigates the impact of the diversity of sources of synthetic data on fine-tuned large language models. We focus on three key dimensions: distribution collapse, adversarial robustness, and self-preference bias. Our findings reveal that fine-tuning models on synthetic data from diverse sources can mitigate distribution collapse, preserving the breadth of the output distribution and the diversity of the output text. Furthermore, while both human and synthetic fine-tuning data can remove safeguards, we observe a tendency for higher output quality in the latter case, thus making outputs potentially more usable and dangerous. Finally, we also find evidence that fine-tuning reduces self-preference bias, with human data being the most effective, followed by multi-source synthetic data.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 28 of its 57 references that are in the map. First posted on arXiv 3 Nov 2025.

Builds on

28 of 57 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025BARE: Leveraging Base Language Models for Few-Shot Synthetic Data Generation3 Feb

Other references · 27

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2024Qwen2.5 Technical Reportcited by 4,713
  8. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  9. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  10. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  11. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
  12. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
15 more
  1. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 431
  2. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 351
  3. 2024Self-Preference Bias in LLM-as-a-Judgecited by 248
  4. 2023Self-Consuming Generative Models Go MADcited by 239
  5. 2024LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Taskscited by 236
  6. 2022Will we run out of data? Limits of LLM scaling based on human-generated datacited by 187
  7. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 183
  8. 2024Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluatorscited by 146
  9. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 127
  10. 2024A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'cited by 114
  11. 2023The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Textcited by 97
  12. 2025Base Models Beat Aligned Models at Randomness and Creativitycited by 48
  13. 2025Position: Model Collapse Does Not Mean What You Thinkcited by 32
  14. 2024The Dark Side of Dataset Scaling: Evaluating Racial Classification in Multimodal Modelscited by 23
  15. 2025Knockout LLM Assessment: Using Large Language Models for Evaluations through Iterative Pairwise Comparisonscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Invent a Dataset: Measuring dataset generation abilities with zero seedcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024On the Diversity of Synthetic Data and its Impact on Training Large Language Models
  2. 2025What Matters in LLM-generated Data: Diversity and Its Effect on Model Fine-Tuning
  3. 2025Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls
  4. 2024Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models
  5. 2025Characterizing Model Behavior Under Synthetic Data Training: An Empirical Study Across Scales and Mixing Ratios
  6. 2024Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspective
  7. 2024How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse
  8. 2025Does Training on Synthetic Data Make Models Less Robust?