mlsanityData to 14 Oct 2026

KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Coding

arXiv 2503.02951 ↗First posted 4 Mar 2025cs.LG cs.AI cs.CLcited by 119

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

We introduce KodCode, a synthetic dataset that addresses the persistent challenge of acquiring high-quality, verifiable training data across diverse difficulties and domains for training Large Language Models for coding. Existing code-focused resources typically fail to ensure either the breadth of coverage (e.g., spanning simple coding tasks to advanced algorithmic problems) or verifiable correctness (e.g., unit tests). In contrast, KodCode comprises question-solution-test triplets that are systematically validated via a self-verification procedure. Our pipeline begins by synthesizing a broad range of coding questions, then generates solutions and test cases with additional attempts allocated to challenging problems. Finally, post-training data synthesis is done by rewriting questions into diverse formats and generating responses under a test-based reject sampling procedure from a reasoning model (DeepSeek R1). This pipeline yields a large-scale, robust and diverse coding dataset. KodCode is suitable for supervised fine-tuning and the paired unit tests also provide great potential for RL tuning. Fine-tuning experiments on coding benchmarks (HumanEval(+), MBPP(+), BigCodeBench, and LiveCodeBench) demonstrate that KodCode-tuned models achieve state-of-the-art performance, surpassing models like Qwen2.5-Coder-32B-Instruct and DeepSeek-R1-Distill-Llama-70B.

Key facts. Cited by 119 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 33 of its 61 references that are in the map. First posted on arXiv 4 Mar 2025.

Builds on

33 of 61 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Qwen2.5-Coder Technical Report18 Sep
  2. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code12 Mar
  3. 2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement22 Feb
  4. 2023Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistency29 Sep
  5. 2022Competition-Level Code Generation with AlphaCode8 Feb

Other references · 28

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2021Program Synthesis with Large Language Modelscited by 3,637
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  7. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  8. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  9. 2023LIMA: Less Is More for Alignmentcited by 1,100
  10. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  11. 2021Measuring Coding Challenge Competence With APPScited by 902
  12. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
16 more
  1. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  2. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 546
  3. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 346
  4. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  5. 2024CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challengescited by 282
  6. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 256
  7. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
  8. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 238
  9. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Datacited by 200
  10. 2023TACO: Topics in Algorithmic COde generation datasetcited by 160
  11. 2024MAmmoTH2: Scaling Instructions from the Webcited by 145
  12. 2024Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-stepcited by 127
  13. 2024SelfCodeAlign: Self-Alignment for Code Generationcited by 62
  14. 2024CodecLM: Aligning Language Models with Tailored Synthetic Datacited by 39
  15. 2024Preference Optimization for Reasoning with Pseudo Feedbackcited by 39
  16. 2024On the Evaluation of Large Language Models in Unit Test Generationcited by 34

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 46; 2026: 73.

119 papers.

Sort:

By area:

  1. 2025Kimi K2: Open Agentic Intelligencecited by 567
  2. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 186
  3. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  4. 2025MemGen: Weaving Generative Latent Memory for Self-Evolving Agentscited by 109
  5. 2025Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Librarycited by 101
  6. 2025Dream-Coder 7B: An Open Diffusion Language Model for Codecited by 81
  7. 2025TOUCAN: Synthesizing 1.5M Tool-Agentic Data from Real-World MCP Environmentscited by 69
  8. 2025Blending Supervised and Reinforcement Fine-Tuning with Prefix Samplingcited by 60
  9. 2025Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzlescited by 56
  10. 2025Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Modelscited by 54
  11. 20251.4 Million Open-Source Distilled Reasoning Dataset to Empower Large Language Model Trainingcited by 49
  12. 2025Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Modelscited by 37
38 more
  1. 2025QiMeng-CodeV-R1: Reasoning-Enhanced Verilog Generationcited by 35
  2. 2025RollPacker: Mitigating Long-Tail Rollouts for Fast, Synchronous RL Post-Trainingcited by 35
  3. 2025Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimizationcited by 33
  4. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 30
  5. 2025Learning Unmasking Policies for Diffusion Language Modelscited by 30
  6. 2025CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewardscited by 29
  7. 2025Not All Correct Answers Are Equal: Why Your Distillation Source Matterscited by 25
  8. 2025OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critiquecited by 25
  9. 2025RL Grokking Recipe: How Does RL Unlock and Transfer New Algorithms in LLMs?cited by 23
  10. 2025SWE-Synth: Synthesizing Verifiable Bug-Fix Data to Enable Large Language Models in Resolving Real-World Bugscited by 22
  11. 2025DeepDistill: Enhancing LLM Reasoning Capabilities via Large-Scale Difficulty-Graded Data Trainingcited by 18
  12. 2025Text2Grad: Reinforcement Learning from Natural Language Feedbackcited by 18
  13. 2025PurpCode: Reasoning for Safer Code Generationcited by 18
  14. 2025Generalizable End-to-End Tool-Use RL with Synthetic CodeGymcited by 17
  15. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
  16. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 15
  17. 2026LatentMem: Customizing Latent Memory for Multi-Agent Systemscited by 15
  18. 2025ReasoningV: Efficient Verilog Code Generation with Adaptive Hybrid Reasoning Modelcited by 12
  19. 2025InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model Fusioncited by 12
  20. 2025MiroMind-M1: An Open-Source Advancement in Mathematical Reasoning via Context-Aware Multi-Stage Policy Optimizationcited by 12
  21. 2026SimCT: Recovering Lost Supervision for Cross-Tokenizer On-Policy Distillationcited by 12
  22. 2025Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoningcited by 11
  23. 2025NP-Engine: Empowering Optimization Reasoning in Large Language Models with Verifiable Synthetic NP Problemscited by 11
  24. 2026Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillationcited by 11
  25. 2025Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoningcited by 10
  26. 2026Learning How to Remember: A Meta-Cognitive Management Method for Structured and Transferable Agent Memorycited by 10
  27. 2026Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Trainingcited by 10
  28. 2025PromptCoT 2.0: Scaling Prompt Synthesis for Large Language Model Reasoningcited by 9
  29. 2025JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligencecited by 9
  30. 2026Harnessing LLM Agents with Skill Programscited by 9
  31. 2025InfiFPO: Implicit Model Fusion via Preference Optimization in Large Language Modelscited by 8
  32. 2026FlashMem: Distilling Intrinsic Latent Memory via Computation Reusecited by 8
  33. 2026Code-A1: Adversarial Evolving of Code LLM and Test LLM via Reinforcement Learningcited by 8
  34. 2026Cross-Tokenizer On-Policy Distillation via Byte-Prefix Marginalizationcited by 8
  35. 2025Critique-Coder: Enhancing Coder Models by Critique Reinforcement Learningcited by 7
  36. 2026LaDi-RL: Latent Diffusion Reasoning Prevents Entropy Collapse in Reinforcement Learningcited by 7
  37. 2026ExecVerify: White-Box RL with Verifiable Stepwise Rewards for Code Execution Reasoningcited by 6
  38. 2025MURPHY: Feedback-Aware GRPO with Retrospective Credit Assignment for Multi-Turn Code Generationcited by 5

50 most cited of 119 All 119 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation
  2. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Code
  3. 2025Increasing LLM Coding Capabilities through Diverse Synthetic Coding Tasks
  4. 2025PuzzleClone: A DSL-Powered Framework for Synthesizing Verifiable Data
  5. 2025rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Dataset
  6. 2025QueST: Incentivizing LLMs to Generate Difficult Problems
  7. 2025AKD : Adversarial Knowledge Distillation For Large Language Models Alignment on Coding tasks
  8. 2024Case2Code: Scalable Synthetic Data for Code Generation