mlsanityData to 14 Oct 2026

GenX: Mastering Code and Test Generation with Execution Feedback

arXiv 2412.13464 ↗First posted 18 Dec 2024cs.SE cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Recent advancements in language modeling have enabled the translation of natural language into code, and the use of execution feedback to improve code generation. However, these methods often rely heavily on pre-existing test cases, which may not always be available or comprehensive. In this work, we propose a novel approach that concurrently trains a code generation model and a test generation model, utilizing execution feedback to refine and enhance the performance of both. We introduce two strategies for test and code data augmentation and a new scoring function for code and test ranking. We experiment on the APPS dataset and demonstrate that our approach can effectively generate and augment test cases, filter and synthesize correct code solutions, and rank the quality of generated code and tests. The results demonstrate that our models, when iteratively trained with an increasing number of test cases and code solutions, outperform those trained on the original dataset.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 23 of its 31 references that are in the map. First posted on arXiv 18 Dec 2024.

Builds on

23 of 31 references.

References · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  5. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  6. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  7. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  8. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  9. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  10. 2023Teaching Large Language Models to Self-Debugcited by 988
  11. 2021Measuring Coding Challenge Competence With APPScited by 902
  12. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
11 more
  1. 2023Textbooks Are All You Needcited by 569
  2. 2022CodeT: Code Generation with Generated Testscited by 384
  3. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  4. 2023An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generationcited by 194
  5. 2017RobustFill: Neural Program Learning under Noisy I/Ocited by 126
  6. 2023RLTF: Reinforcement Learning from Unit Test Feedbackcited by 114
  7. 2023Self-Edit: Fault-Aware Code Editor for Code Generationcited by 111
  8. 2023Execution-based Code Generation using Deep Reinforcement Learningcited by 99
  9. 2018Leveraging Grammar and Reinforcement Learning for Neural Program Synthesiscited by 84
  10. 2024If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agentscited by 79
  11. 2023Automatic Unit Test Data Generation and Actor-Critic Reinforcement Learning for Code Synthesiscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025DARS: Dynamic Action Re-Sampling to Enhance Coding Agent Performance by Adaptive Tree Traversalcited by 20
  2. 2025Execution Guided Line-by-Line Code Generationcited by 10
  3. 2025CodeChemist: Test-Time Scaling for Low-Resource Code Generation via Functional Knowledge Transfercited by 1
  4. 2026Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoringcited by 1