mlsanityData to 14 Oct 2026

CodeLutra: Boosting LLM Code Generation via Preference-Guided Refinement

arXiv 2411.05199 ↗First posted 7 Nov 2024cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have revolutionized code generation but require significant resources and often over-generalize, limiting their task-specific efficiency. Fine-tuning smaller, open-source LLMs provides a cost-effective alternative. However, standard supervised approaches rely only on correct examples, missing valuable insights from failures. We introduce CodeLutra, a framework that leverages both correct and incorrect code attempts. Instead of using only correct solutions, CodeLutra applies iterative preference-based refinement, comparing successful and failed outputs to better approximate desired results. This approach narrows the performance gap with state-of-the-art larger models without requiring massive datasets or auxiliary models. For instance, on a challenging data science coding task, using only 500 samples improved Llama-3-8B's accuracy from 28.2% to 48.6%, approaching GPT-4's level. By learning from both successes and mistakes, CodeLutra provides a scalable and efficient path to high-quality code generation, making smaller open-source models more competitive with leading closed-source alternatives.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 42 of its 46 references that are in the map. First posted on arXiv 7 Nov 2024.

Builds on

42 of 46 references.

References · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  8. 2017Deep reinforcement learning from human preferencescited by 3,912
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  10. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  11. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  12. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
30 more
  1. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  2. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  3. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  4. 2023Teaching Large Language Models to Self-Debugcited by 988
  5. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  6. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
  7. 2023StarCoder: may the source be with you!cited by 799
  8. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  9. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
  10. 2024Self-Rewarding Language Modelscited by 681
  11. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  12. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 502
  13. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 452
  14. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  15. 2023Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraintcited by 350
  16. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 333
  17. 2023Statistical Rejection Sampling Improves Preference Optimizationcited by 323
  18. 2024Iterative Reasoning Preference Optimizationcited by 246
  19. 2024Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positivecited by 241
  20. 2024Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learningcited by 241
  21. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 189
  22. 2022ReACC: A Retrieval-Augmented Code Completion Frameworkcited by 119
  23. 2024Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizercited by 109
  24. 2023Execution-based Code Generation using Deep Reinforcement Learningcited by 99
  25. 2022Less Training, More Repairing Please: Revisiting Automated Program Repair via Zero-shot Learningcited by 76
  26. 2023PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedbackcited by 69
  27. 2024Towards Analyzing and Understanding the Limitations of DPO: A Theoretical Perspectivecited by 69
  28. 2024NExT: Teaching Large Language Models to Reason about Code Executioncited by 63
  29. 2023InstructCoder: Instruction Tuning Large Language Models for Code Editingcited by 29
  30. 2022Overwatch: Learning Patterns in Code Edit Sequencescited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 1.

6 papers.

  1. 2025Sailing by the Stars: A Survey on Reward Models and Learning Strategies for Learning from Rewardscited by 14
  2. 2025Boosting Chart-to-Code Generation in MLLM via Dual Preference-Guided Refinementcited by 11
  3. 2025RefineCoder: Iterative Improving of Large Language Models via Adaptive Critique Refinement for Code Generationcited by 8
  4. 2025Curiosity by Design: An LLM-based Coding Assistant Asking Clarification Questionscited by 4
  5. 2026ExecCritic: Learn to Test, Test to Improve for Coding Agentscited by 3
  6. 2025Helmsman: Autonomous Synthesis of Federated Learning Systems via Collaborative LLM Agentscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024CodeUltraFeedback: An LLM-as-a-Judge Dataset for Aligning Large Language Models to Coding Preferences
  2. 2025LeetCodeDataset: A Temporal Dataset for Robust Evaluation and Efficient Training of Code LLMs
  3. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Code
  4. 2025Data-efficient LLM Fine-tuning for Code Generation
  5. 2024DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories
  6. 2024Code Less, Align More: Efficient LLM Fine-tuning for Code Generation with Data Pruning
  7. 2025CodeBoost: Boosting Code LLMs by Squeezing Knowledge from Code Snippets with RL
  8. 2025BigCodeArena: Unveiling More Reliable Human Preferences in Code Generation via Execution