mlsanityData to 7 Oct 2026

Improving Code Generation by Training with Natural Language Feedback

arXiv 2303.16749 ↗First posted 28 Mar 2023cs.SE cs.AI cs.CLcited by 75

Not on the map. Why?

Cited by 21 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The potential for pre-trained large language models (LLMs) to use natural language feedback at inference time has been an exciting recent development. We build upon this observation by formalizing an algorithm for learning from natural language feedback at training time instead, which we call Imitation learning from Language Feedback (ILF). ILF requires only a small amount of human-written feedback during training and does not require the same feedback at test time, making it both user-friendly and sample-efficient. We further show that ILF can be seen as a form of minimizing the KL divergence to the ground truth distribution and demonstrate a proof-of-concept on a neural program synthesis task. We use ILF to improve a Codegen-Mono 6.1B model's pass@1 rate by 38% relative (and 10% absolute) on the Mostly Basic Python Problems (MBPP) benchmark, outperforming both fine-tuning on MBPP and fine-tuning on repaired programs written by humans. Overall, our results suggest that learning from human-written natural language feedback is both more effective and sample-efficient than training exclusively on demonstrations for improving an LLM's performance on code generation tasks.

Key facts. Cited by 75 papers in the map (data through 7 Oct 2026). Builds on 28 of its 36 references that are in the map. First posted on arXiv 28 Mar 2023.

Builds on

28 of 36 references.

References · 28

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2021Program Synthesis with Large Language Modelscited by 3,509
  7. 2020Learning to summarize from human feedbackcited by 2,562
  8. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  9. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  10. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  11. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  12. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
16 more
  1. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  2. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  3. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  4. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  5. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  6. 2022Can language models learn from explanations in context?cited by 248
  7. 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 212
  8. 2019SPoC: Search-based Pseudocode to Codecited by 176
  9. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  10. 2022Large Language Models are Few-shot Testers: Exploring LLM-based General Bug Reproductioncited by 110
  11. 2022Productivity Assessment of Neural Code Completioncited by 97
  12. 2020Evaluating Explanations: How much do explanations from the teacher aid students?cited by 94
  13. 2022CERT: Continual Pre-Training on Sketches for Library-Oriented Code Generationcited by 87
  14. 2021When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Datacited by 75
  15. 2022Training Language Models with Language Feedbackcited by 48
  16. 2020Speak to your Parser: Interactive Text-to-SQL with Natural Language Feedbackcited by 44

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 29; 2024: 25; 2025: 15; 2026: 6.

75 papers.

Sort:

By area:

  1. 2023Teaching Large Language Models to Self-Debugcited by 963
  2. 2023Large Language Models as Optimizerscited by 830
  3. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  4. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  5. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
  6. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  7. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 257
  8. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  9. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 229
  10. 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learningcited by 155
  11. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 146
  12. 2023Generalized Planning in PDDL Domains with Pretrained Large Language Modelscited by 139
38 more
  1. 2024Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-stepcited by 123
  2. 2024GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinementscited by 101
  3. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 89
  4. 2024If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agentscited by 77
  5. 2023Large Language Models for Supply Chain Optimizationcited by 61
  6. 2023GPT4AIGChip: Towards Next-Generation AI Accelerator Design Automation via Large Language Modelscited by 59
  7. 2023CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modulescited by 58
  8. 2023CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translationcited by 55
  9. 2023Learning to Generate Better Than Your LLMcited by 49
  10. 2024Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategiescited by 44
  11. 2023A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoningcited by 41
  12. 2024LeDex: Training LLMs to Better Self-Debug and Explain Codecited by 39
  13. 2024CodeTree: Agent-guided Tree Search for Code Generation with Large Language Modelscited by 32
  14. 2023CodeCoT: Tackling Code Syntax Errors in CoT Reasoning for Code Generationcited by 29
  15. 2023LeTI: Learning to Generate from Textual Interactionscited by 25
  16. 2024Can We Rely on LLM Agents to Draft Long-Horizon Plans? Let's Take TravelPlanner as an Examplecited by 22
  17. 2025Sample, Don't Search: Rethinking Test-Time Alignment for Language Modelscited by 21
  18. 2023Exploring and Characterizing Large Language Models For Embedded System Development and Debuggingcited by 20
  19. 2023Enhancing Network Management Using Code Generated by Large Language Modelscited by 20
  20. 2023AI-native Interconnect Framework for Integration of Large Language Model Technologies in 6G Systemscited by 19
  21. 2024Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineeringcited by 16
  22. 2025HelpSteer3: Human-Annotated Feedback and Edit Data to Empower Inference-Time Scaling in Open-Ended General-Domain Taskscited by 16
  23. 2024Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Modelscited by 15
  24. 2023FRANC: A Lightweight Framework for High-Quality Code Generationcited by 14
  25. 2024Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmerscited by 14
  26. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challengescited by 14
  27. 2025Imitation Learning for Multi-turn LM Agents via On-policy Expert Correctionscited by 14
  28. 2023Explain-then-Translate: An Analysis on Improving Program Translation with Self-generated Explanationscited by 12
  29. 2023Nova: Generative Language Models for Assembly Code with Hierarchical Attention and Contrastive Learningcited by 12
  30. 2025LLM4EFFI: Leveraging Large Language Models to Enhance Code Efficiency and Correctnesscited by 10
  31. 2023Pitfalls in Language Models for Code Intelligence: A Taxonomy and Surveycited by 9
  32. 2024Teaching Language Models to Self-Improve by Learning from Language Feedbackcited by 9
  33. 2024LATTE: Improving Latex Recognition for Tables and Formulae with Iterative Refinementcited by 8
  34. 2024Physics Reasoner: Knowledge-Augmented Reasoning for Solving Physics Problems with Large Language Modelscited by 8
  35. 2026ShieldedCode: Learning Robust Representations for Virtual Machine Protected Codecited by 8
  36. 2023Personalised Distillation: Empowering Open-Sourced LLMs with Adaptive Learning for Code Generationcited by 7
  37. 2023Evaluating the Potential of Leading Large Language Models in Reasoning Biology Questionscited by 7
  38. 2024INDICT: Code Generation with Internal Dialogues of Critiques for Both Security and Helpfulnesscited by 7

50 most cited of 75 All 75 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Training Language Models with Language Feedback at Scale
  2. 2025Teaching Language Models to Critique via Reinforcement Learning
  3. 2023CodeFusion: A Pre-trained Diffusion Model for Code Generation
  4. 2023PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedback
  5. 2023Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation
  6. 2025Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation
  7. 2023Self-Refine: Iterative Refinement with Self-Feedback
  8. 2024Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Models