mlsanityData to 7 Oct 2026

Code and instruction data synthesis

106 papers1 this weekMedium confidence in this grouping: mostly one topic, with blurry edgesMedium confidence in this grouping: mostly one topic, with blurry edges · About

Keywords: code, instruction, llms, language, generation

Key facts. Code and instruction data synthesis: 106 papers on arXiv since 2022 (data through 7 Oct 2026). 39 of its 106 mapped papers (37%) cite “Self-Instruct: Aligning Language Models with Self-Generated Instructions” (2022, arXiv 2212.10560). 7 of its 106 mapped papers (7%) cite “Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models” (2024, arXiv 2402.13064). 1 new paper 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

1 new since 17 Sep

  1. Turnslide: Scalable Multi-Turn Data Synthesis by Walking a Finite-State Machine

Foundations

Most cited by papers here

  1. Self-Instruct: Aligning Language Models with Self-Generated Instructions2022cited by 39 here
  2. Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models2024cited by 7 here
  3. CodecLM: Aligning Language Models with Tailored Synthetic Data2024cited by 3 here
  4. Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation2025cited by 3 here
  5. KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Coding2025cited by 3 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 3 here
  2. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 2 here
  3. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 2 here
  4. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 6 here
  5. 2024Self-Rewarding Language Modelscited by 3 here
  6. 2024Refined Direct Preference Optimization with Synthetic Data for Behavioral Alignment of LLMscited by 2 here
18 more it builds on, not in this chain
  1. 2022Large Language Models Can Self-Improvecited by 6 here
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 39 here
  3. 2023The False Promise of Imitating Proprietary LLMscited by 5 here
  4. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 3 here
  5. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 2 here
  6. 2024Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Modelscited by 7 here
  7. 2024Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferencescited by 2 here
  8. 2024How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapsecited by 2 here
  9. 2024CodecLM: Aligning Language Models with Tailored Synthetic Datacited by 3 here
  10. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 11 here
  11. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 4 here
  12. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 4 here
  13. 2024Evaluating Language Models as Synthetic Data Generatorscited by 3 here
  14. 2025Learning to Solve and Verify: A Self-Play Framework for Code and Test Generationcited by 3 here
  15. 2025KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Codingcited by 3 here
  16. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Codecited by 4 here
  17. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 4 here
  18. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 3 here

The 24 most cited of 96.

Browse

30 of 106, ranked 6–35

  1. 6Instruction-Tuning Data Synthesis from Scratch via Web Reconstruction20252
  2. 7Code-Optimise: Self-Generated Preference Data for Correctness and Efficiency20242
  3. 8Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Models20242
  4. 9SPaR: Self-Play with Tree-Search Refinement to Improve Instruction-Following in Large Language Models20242
  5. 10CodePivot: Bootstrapping Multilingual Transpilation in LLMs via Reinforcement Learning without Parallel Corpora20261
  6. 11Revisit Self-Debugging with Self-Generated Tests for Code Generation20251
  7. 12Improving Natural Language Understanding for LLMs via Large-Scale Instruction Synthesis20251
  8. 13A Data Synthesis Method Driven by Large Language Models for Proactive Mining of Implicit User Intentions in Tourism20251
  9. 14Knowledge-Graph-Driven Data Synthesis for Low-Resource Software Development: A HarmonyOS Case Study20251
  10. 15SynthDST: Synthetic Data is All You Need for Few-Shot Dialog State Tracking20241
All 30
  1. 16Mosaic-IT: Cost-Free Compositional Data Synthesis for Instruction Tuning20241
  2. 17Case2Code: Scalable Synthetic Data for Code Generation20241
  3. 18COAST: Enhancing the Code Debugging Ability of LLMs through Communicative Agent Based Data Synthesis20241
  4. 19DSTC: Direct Preference Learning with Only Self-Generated Tests and Code to Improve Code LMs20241
  5. 20Becoming self-instruct: introducing early stopping criteria for minimal instruct tuning20231
  6. 21Explain-then-Translate: An Analysis on Improving Program Translation with Self-generated Explanations20231
  7. 22Bootstrapping Code Translation with Weighted Multilanguage Exploration20260
  8. 23FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale20260
  9. 24EvoCodeBench: A Human-Performance Benchmark for Self-Evolving LLM-Driven Coding Systems20260
  10. 25Pedagogically-Inspired Data Synthesis for Language Model Knowledge Distillation20260
  11. 26Unseen-Codebases-Domain Data Synthesis and Training Based on Code Graphs20260
  12. 27Steerable Instruction Following Coding Data Synthesis with Actor-Parametric Schema Co-Evolution20260
  13. 28DS2-Instruct: Domain-Specific Data Synthesis for Large Language Models Instruction Tuning20260
  14. 29QiMeng-CodeV-SVA: Training Specialized LLMs for Hardware Assertion Generation via RTL-Grounded Bidirectional Data Synthesis20260
  15. 30A Deep Dive into Scaling RL for Code Generation with Synthetic Data and Curricula20260
  16. 31Self-Improving Code Generation via Semantic Entropy and Behavioral Consensus20260
  17. 32Task-Aware LLM Routing with Multi-Level Task-Profile-Guided Data Synthesis for Cold-Start Scenarios20260
  18. 33Synthesizing Instruction-Tuning Datasets with Contrastive Decoding20260
  19. 34Syn-TurnTurk: A Synthetic Dataset for Turn-Taking Prediction in Turkish Dialogues20260
  20. 35ACE: Self-Evolving LLM Coding Framework via Adversarial Unit Test Generation and Preference Optimization20260