mlsanityData to 14 Oct 2026

Towards Active Synthetic Data Generation for Finetuning Language Models

arXiv 2512.00884 ↗First posted 30 Nov 2025cs.LG cs.CLcited by 6

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

A common and effective means for improving language model capabilities involves finetuning a ``student'' language model's parameters on generations from a more proficient ``teacher'' model. Termed ``synthetic data'', these generations are often produced before any student finetuning, but some work has considered generating new synthetic samples as training progresses. This paper studies and advocates for the latter case, where data are generated in an iterative, closed-loop fashion that is guided by the current state of the student model. For a fixed budget of generated samples, or a budget in terms of compute spent querying a teacher, we show that this curation of finetuning data affords improved student performance over static generation. Further, while there have been several LLM-specific methods proposed that operate in this regime, we find that simple, inexpensive selection criteria from the active learning literature tend to be most performant. We validate these claims across four mathematical and logical reasoning datasets using four different small language models.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 53 of its 71 references that are in the map. First posted on arXiv 30 Nov 2025.

Builds on

53 of 71 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale Pretraining14 Aug

Other references · 52

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  6. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  9. 2023Mistral 7Bcited by 3,164
  10. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  11. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  12. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
38 more
  1. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  2. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  3. 2017Deep Bayesian Active Learning with Image Datacited by 819
  4. 2023Instruction Tuning with GPT-4cited by 686
  5. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  6. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 629
  7. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
  8. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 610
  9. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 499
  10. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 432
  11. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 364
  12. 2019Selection via Proxy: Efficient Data Selection for Deep Learningcited by 339
  13. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 337
  14. 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 328
  15. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 307
  16. 2025Small Language Models are the Future of Agentic AIcited by 270
  17. 2022Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learntcited by 236
  18. 2024Orca-Math: Unlocking the potential of SLMs in Grade School Mathcited by 151
  19. 2024OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Datasetcited by 143
  20. 2015Online Batch Selection for Faster Training of Neural Networkscited by 122
  21. 2024Self-Improvement in Language Models: The Sharpening Mechanismcited by 104
  22. 2019Accelerating Deep Learning by Focusing on the Biggest Loserscited by 94
  23. 2021Gone Fishing: Neural Active Learning with Fisher Embeddingscited by 88
  24. 2023Loose lips sink ships: Mitigating Length Bias in Reinforcement Learning from Human Feedbackcited by 78
  25. 2024LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancementcited by 70
  26. 2023LAVA: Data Valuation without Pre-Specified Learning Algorithmscited by 69
  27. 2024Synthetic continued pretrainingcited by 60
  28. 2023Instruction Mining: Instruction Data Selection for Tuning Large Language Modelscited by 58
  29. 2025Overtrained Language Models Are Harder to Fine-Tunecited by 58
  30. 2024Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the datacited by 51
  31. 2023No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Modelscited by 41
  32. 2023Forward-Backward Reasoning in Large Language Models for Mathematical Verificationcited by 40
  33. 2023Lion: Adversarial Distillation of Proprietary Large Language Modelscited by 34
  34. 2023Bad Students Make Great Teachers: Active Learning Accelerates Large-Scale Visual Understandingcited by 28
  35. 2024Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models -- The Story Goes Oncited by 24
  36. 2023Streaming Active Learning with Deep Neural Networkscited by 20
  37. 2024ELAD: Explanation-Guided Large Language Models Active Distillationcited by 12
  38. 2024Evolving Knowledge Distillation with Large Language Models and Active Learningcited by 12

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Exploring Autonomous Agentic Data Engineering for Model Specializationcited by 2
  2. 2026When the Strongest Teacher Is Not the Best Teacher: Student-Centric Answer Selectioncited by 1
  3. 2026What Drives Interactive Improvement from Feedback?cited by 1
  4. 2026QVAC Genesis III: A Large-Scale, High-Quality Open Synthetic STEM Corpus for Efficient Language Model Pre-Trainingcited by 0
  5. 2026AutoDataBench: Can Agents Write the Data That Feeds the Self-Improvement Loop?cited by 0
  6. 2026G"odel Forest: Balancing Search Depth and Breadth for Data-Centric Recursive Self-Improvementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Evaluating Language Models as Synthetic Data Generators
  2. 2023Synthetic Data Generation in Low-Resource Settings via Fine-Tuning of Large Language Models
  3. 2025Synthetic Function Demonstrations Improve Generation in Low-Resource Programming Languages
  4. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Code
  5. 2025BARE: Leveraging Base Language Models for Few-Shot Synthetic Data Generation
  6. 2024A Survey on Data Synthesis and Augmentation for Large Language Models
  7. 2024Balancing Cost and Effectiveness of Synthetic Data Generation Strategies for LLMs
  8. 2026Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation