# Code-Optimise: Self-Generated Preference Data for Correctness and Efficiency

arXiv: https://arxiv.org/abs/2406.12502 · HTML: https://mlsanity.com/paper/2406.12502/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Code and instruction data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 19 of its 27 references that are in the map.
- First posted on arXiv 18 Jun 2024.

## Placed in

- Synthetic data and self-improvement for LLMs / [Code and instruction data synthesis](https://mlsanity.com/synth/code-and-instruction-data-synthesis/)

## Builds on (most cited first, 19 of 19)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [A General Theoretical Paradigm to Understand Learning from Human Preferences](https://mlsanity.com/paper/2310.12036/) (2023)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [WizardCoder: Empowering Code Large Language Models with Evol-Instruct](https://mlsanity.com/paper/2306.08568/) (2023)
- [ORPO: Monolithic Preference Optimization without Reference Model](https://mlsanity.com/paper/2403.07691/) (2024)
- [StarCoder 2 and The Stack v2: The Next Generation](https://mlsanity.com/paper/2402.19173/) (2024)
- [RRHF: Rank Responses to Align Language Models with Human Feedback without tears](https://mlsanity.com/paper/2304.05302/) (2023)
- [UltraFeedback: Boosting Language Models with Scaled AI Feedback](https://mlsanity.com/paper/2310.01377/) (2023)
- [SLiC-HF: Sequence Likelihood Calibration with Human Feedback](https://mlsanity.com/paper/2305.10425/) (2023)
- [The Stack: 3 TB of permissively licensed source code](https://mlsanity.com/paper/2211.15533/) (2022)
- [Statistical Rejection Sampling Improves Preference Optimization](https://mlsanity.com/paper/2309.06657/) (2023)
- [Compilable Neural Code Generation with Compiler Feedback](https://mlsanity.com/paper/2203.05132/) (2022)
- [Automatic Unit Test Data Generation and Actor-Critic Reinforcement Learning for Code Synthesis](https://mlsanity.com/paper/2310.13669/) (2023)

## Built on by (most cited first, 16 of 16)

- [Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step](https://mlsanity.com/paper/2501.13926/) (2025)
- [A Survey of Direct Preference Optimization](https://mlsanity.com/paper/2503.11701/) (2025)
- [ECCO: Can We Improve Model-Generated Code Efficiency Without Sacrificing Functional Correctness?](https://mlsanity.com/paper/2407.14044/) (2024)
- [A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applications](https://mlsanity.com/paper/2410.15595/) (2024)
- [In Line with Context: Repository-Level Code Generation via Context Inlining](https://mlsanity.com/paper/2601.00376/) (2026)
- [SEAlign: Alignment Training for Software Engineering Agent](https://mlsanity.com/paper/2503.18455/) (2025)
- [Teaching Your Models to Understand Code via Focal Preference Alignment](https://mlsanity.com/paper/2503.02783/) (2025)
- [SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Masks](https://mlsanity.com/paper/2410.05102/) (2024)
- [DSTC: Direct Preference Learning with Only Self-Generated Tests and Code to Improve Code LMs](https://mlsanity.com/paper/2411.13611/) (2024)
- [Alignment with Fill-In-the-Middle for Enhancing Code Generation](https://mlsanity.com/paper/2508.19532/) (2025)
- [Learning to Align Human Code Preferences](https://mlsanity.com/paper/2507.20109/) (2025)
- [Self-Improving Code Generation via Semantic Entropy and Behavioral Consensus](https://mlsanity.com/paper/2603.29292/) (2026)
- [CROP: Expert-Aligned Image Cropping via Compositional Reasoning and Optimizing Preference](https://mlsanity.com/paper/2605.12545/) (2026)
- [EffiPair: Improving the Efficiency of LLM-generated Code with Differential Execution Feedback](https://mlsanity.com/paper/2604.05137/) (2026)
- [Improving Code Translation with Syntax-Guided and Semantic-aware Preference Optimization](https://mlsanity.com/paper/2605.13229/) (2026)
- [Bridging Functional Correctness and Runtime Efficiency Gaps in LLM-Based Code Translation](https://mlsanity.com/paper/2606.17683/) (2026)

## Abstract (arXiv)

Code Language Models have been trained to generate accurate solutions, typically with no regard for runtime. On the other hand, previous works that explored execution optimisation have observed corresponding drops in functional correctness. To that end, we introduce Code-Optimise, a framework that incorporates both correctness (passed, failed) and runtime (quick, slow) as learning signals via self-generated preference data. Our framework is both lightweight and robust as it dynamically selects solutions to reduce overfitting while avoiding a reliance on larger models for learning signals. Code-Optimise achieves significant improvements in pass@k while decreasing the competitive baseline runtimes by an additional 6% for in-domain data and up to 3% for out-of-domain data. As a by-product, the average length of the generated solutions is reduced by up to 48% on MBPP and 23% on HumanEval, resulting in faster and cheaper inference. The generated data and codebase is open-sourced at https://github.com/huawei-noah/HEBO/tree/Code_Optimise.