# Code and instruction data synthesis

Area: [Synthetic data and self-improvement for LLMs](https://mlsanity.com/synth/index.md). HTML: https://mlsanity.com/synth/code-and-instruction-data-synthesis/

## Key facts

- Code and instruction data synthesis: 106 papers on arXiv since 2022 (data through 7 Oct 2026).
- 39 of its 106 mapped papers (37%) cite “Self-Instruct: Aligning Language Models with Self-Generated Instructions” (2022, arXiv 2212.10560).
- 7 of its 106 mapped papers (7%) cite “Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models” (2024, arXiv 2402.13064).
- 1 new papers 17 Sep–7 Oct.

## Most cited here

- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022), cited by 39 papers here
- [Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models](https://mlsanity.com/paper/2402.13064/) (2024), cited by 7 papers here
- [CodecLM: Aligning Language Models with Tailored Synthetic Data](https://mlsanity.com/paper/2404.05875/) (2024), cited by 3 papers here
- [Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation](https://mlsanity.com/paper/2502.14948/) (2025), cited by 3 papers here
- [KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Coding](https://mlsanity.com/paper/2503.02951/) (2025), cited by 3 papers here
- [Code-Optimise: Self-Generated Preference Data for Correctness and Efficiency](https://mlsanity.com/paper/2406.12502/) (2024), cited by 2 papers here
- [Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Models](https://mlsanity.com/paper/2406.13542/) (2024), cited by 2 papers here
- [SPaR: Self-Play with Tree-Search Refinement to Improve Instruction-Following in Large Language Models](https://mlsanity.com/paper/2412.11605/) (2024), cited by 2 papers here
- [Instruction-Tuning Data Synthesis from Scratch via Web Reconstruction](https://mlsanity.com/paper/2504.15573/) (2025), cited by 2 papers here
- [Becoming self-instruct: introducing early stopping criteria for minimal instruct tuning](https://mlsanity.com/paper/2307.03692/) (2023), cited by 1 papers here

## New (17 Sep–7 Oct)

- [Turnslide: Scalable Multi-Turn Data Synthesis by Walking a Finite-State Machine](https://mlsanity.com/paper/2610.07070/) (2026-10-05)