mlsanityData to 14 Oct 2026

Code-Optimise: Self-Generated Preference Data for Correctness and Efficiency

arXiv 2406.12502 ↗First posted 18 Jun 2024cs.CLcited by 16

On the map: Synthetic data and self-improvement for LLMs → Code and instruction data synthesis

Abstract (arXiv)

Code Language Models have been trained to generate accurate solutions, typically with no regard for runtime. On the other hand, previous works that explored execution optimisation have observed corresponding drops in functional correctness. To that end, we introduce Code-Optimise, a framework that incorporates both correctness (passed, failed) and runtime (quick, slow) as learning signals via self-generated preference data. Our framework is both lightweight and robust as it dynamically selects solutions to reduce overfitting while avoiding a reliance on larger models for learning signals. Code-Optimise achieves significant improvements in pass@k while decreasing the competitive baseline runtimes by an additional 6% for in-domain data and up to 3% for out-of-domain data. As a by-product, the average length of the generated solutions is reduced by up to 48% on MBPP and 23% on HumanEval, resulting in faster and cheaper inference. The generated data and codebase is open-sourced at https://github.com/huawei-noah/HEBO/tree/Code_Optimise.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Code and instruction data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 19 of its 27 references that are in the map. First posted on arXiv 18 Jun 2024.

Builds on

19 of 27 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec

Other references · 18

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  6. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  7. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 912
  8. 2023StarCoder: may the source be with you!cited by 799
  9. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  10. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
  11. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 502
  12. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 463
6 more
  1. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 406
  2. 2023SLiC-HF: Sequence Likelihood Calibration with Human Feedbackcited by 401
  3. 2022The Stack: 3 TB of permissively licensed source codecited by 359
  4. 2023Statistical Rejection Sampling Improves Preference Optimizationcited by 323
  5. 2022Compilable Neural Code Generation with Compiler Feedbackcited by 47
  6. 2023Automatic Unit Test Data Generation and Actor-Critic Reinforcement Learning for Code Synthesiscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 6; 2026: 6.

16 papers.

Sort:

By area:

  1. 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Stepcited by 131
  2. 2025A Survey of Direct Preference Optimizationcited by 36
  3. 2024ECCO: Can We Improve Model-Generated Code Efficiency Without Sacrificing Functional Correctness?cited by 27
  4. 2024A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applicationscited by 27
  5. 2026In Line with Context: Repository-Level Code Generation via Context Inliningcited by 18
  6. 2025SEAlign: Alignment Training for Software Engineering Agentcited by 11
  7. 2025Teaching Your Models to Understand Code via Focal Preference Alignmentcited by 10
  8. 2024SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Maskscited by 9
  9. 2024DSTC: Direct Preference Learning with Only Self-Generated Tests and Code to Improve Code LMscited by 9
  10. 2025Alignment with Fill-In-the-Middle for Enhancing Code Generationcited by 4
  11. 2025Learning to Align Human Code Preferencescited by 1
  12. 2026Self-Improving Code Generation via Semantic Entropy and Behavioral Consensuscited by 1
4 more
  1. 2026CROP: Expert-Aligned Image Cropping via Compositional Reasoning and Optimizing Preferencecited by 1
  2. 2026EffiPair: Improving the Efficiency of LLM-generated Code with Differential Execution Feedbackcited by 0
  3. 2026Improving Code Translation with Syntax-Guided and Semantic-aware Preference Optimizationcited by 0
  4. 2026Bridging Functional Correctness and Runtime Efficiency Gaps in LLM-Based Code Translationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Mercury: A Code Efficiency Benchmark for Code Large Language Models
  2. 2024Learning Code Preference via Synthetic Evolution
  3. 2025Towards Better Correctness and Efficiency in Code Generation
  4. 2024Thinking Before Running! Efficient Code Generation with Thorough Exploration and Optimal Refinement
  5. 2025Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization
  6. 2024PLUM: Improving Code LMs with Execution-Guided On-Policy Preference Learning Driven By Synthetic Test Cases
  7. 2025Focused-DPO: Enhancing Code Generation Through Focused Preference Optimization on Error-Prone Points
  8. 2026Efficiency Hallucination: Formalizing and Measuring Behavioral Calibration in LLM-Based Code Optimization