mlsanityData to 7 Oct 2026

Aligning Crowd-sourced Human Feedback for Reinforcement Learning on Code Generation by Large Language Models

arXiv 2503.15129 ↗First posted 19 Mar 2025cs.AIcited by 3

On the map: LLM agents and tool use → Software engineering coding agents

also in 1 more
Abstract (arXiv)

This paper studies how AI-assisted programming and large language models (LLM) improve software developers' ability via AI tools (LLM agents) like Github Copilot and Amazon CodeWhisperer, while integrating human feedback to enhance reinforcement learning (RLHF) with crowd-sourced computation to enhance text-to-code generation. Additionally, we demonstrate that our Bayesian optimization framework supports AI alignment in code generation by distributing the feedback collection burden, highlighting the value of collecting human feedback of good quality. Our empirical evaluations demonstrate the efficacy of this approach, showcasing how LLM agents can be effectively trained for improved text-to-code generation. Our Bayesian optimization framework can be designed for general domain-specific languages, promoting the alignment of large language model capabilities with human feedback in AI-assisted programming for code generation.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 47 of its 78 references that are in the map. First posted on arXiv 19 Mar 2025.

Builds on

47 of 78 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  9. 2017Deep reinforcement learning from human preferencescited by 3,846
  10. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  11. 2021Program Synthesis with Large Language Modelscited by 3,509
  12. 2023Qwen Technical Reportcited by 3,273
35 more
  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  2. 2020Learning to summarize from human feedbackcited by 2,562
  3. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  4. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  5. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  6. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  7. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  8. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  9. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
  10. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  11. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  12. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  13. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  14. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 484
  15. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  16. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  17. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 294
  18. 2018code2vec: Learning Distributed Representations of Codecited by 249
  19. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 244
  20. 2017A Survey of Machine Learning for Big Code and Naturalnesscited by 220
  21. 2018code2seq: Generating Sequences from Structured Representations of Codecited by 177
  22. 2024CodeGemma: Open Code Models Based on Gemmacited by 145
  23. 2022RL with KL penalties is better viewed as Bayesian inferencecited by 130
  24. 2023CodeGen2: Lessons for Training LLMs on Programming and Natural Languagescited by 124
  25. 2023Execution-based Code Generation using Deep Reinforcement Learningcited by 95
  26. 2017Abstract Syntax Networks for Code Generation and Semantic Parsingcited by 92
  27. 2023Improving Code Generation by Training with Natural Language Feedbackcited by 75
  28. 2023Inverse Preference Learning: Preference-based RL without a Reward Functioncited by 68
  29. 2023Direct Preference-based Policy Optimization without Reward Modelingcited by 36
  30. 2015On the "Naturalness" of Buggy Codecited by 35
  31. 2023Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Reviewcited by 32
  32. 2023Aligning Language Models with Human Preferences via a Bayesian Approachcited by 27
  33. 2024ChatGLM-RLHF: Practices of Aligning Large Language Models with Human Feedbackcited by 25
  34. 2023Copilot for Xcode: Exploring AI-Assisted Programming by Prompting Cloud-based Large Language Modelscited by 11
  35. 2023Large Language Model-Driven Classroom Flipping: Empowering Student-Centric Peer Questioning with Flipped Interactioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025From Code Generation to Software Testing: AI Copilot with Context-Based RAGcited by 2
  2. 2025Future-Proofing Programmers: Optimal Knowledge Tracing for AI-Assisted Personalized Educationcited by 1
  3. 2026Nemobot Games: Crafting Strategic AI Gaming Agents for Interactive Learning with Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning
  2. 2025ACE-RLHF: Automated Code Evaluation and Socratic Feedback Generation Tool using Large Language Models and Reinforcement Learning with Human Feedback
  3. 2025BigCodeArena: Unveiling More Reliable Human Preferences in Code Generation via Execution
  4. 2024CHAI for LLMs: Improving Code-Mixed Translation in Large Language Models through Reinforcement Learning with AI Feedback
  5. 2025Enhancing LLM Code Generation: A Systematic Evaluation of Multi-Agent Collaboration and Runtime Debugging for Improved Accuracy, Reliability, and Latency
  6. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence
  7. 2024Evaluating and Aligning CodeLLMs on Human Preference
  8. 2024StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback