mlsanityData to 14 Oct 2026

ReflectionCoder: Learning from Reflection Sequence for Enhanced One-off Code Generation

arXiv 2405.17057 ↗First posted 27 May 2024cs.CL cs.AIcited by 14

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Code generation plays a crucial role in various tasks, such as code auto-completion and mathematical reasoning. Previous work has proposed numerous methods to enhance code generation performance, including integrating feedback from the compiler. Inspired by this, we present ReflectionCoder, a novel approach that effectively leverages reflection sequences constructed by integrating compiler feedback to improve one-off code generation performance. Furthermore, we propose reflection self-distillation and dynamically masked distillation to effectively utilize these reflection sequences. Extensive experiments on three benchmarks, i.e., HumanEval (+), MBPP (+), and MultiPL-E, demonstrate that models fine-tuned with our method achieve state-of-the-art performance. Beyond the code domain, we believe this approach can benefit other domains that focus on final results and require long reasoning paths. Code and data are available at https://github.com/SenseLLM/ReflectionCoder.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 50 of its 73 references that are in the map. First posted on arXiv 27 May 2024.

Builds on

50 of 73 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code12 Mar
  2. 2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement22 Feb
  3. 2024StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback2 Feb
  4. 2023Teaching Large Language Models to Self-Debug11 Apr

Other references · 46

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  9. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  11. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  12. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
34 more
  1. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  2. 2021Program Synthesis with Large Language Modelscited by 3,637
  3. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  4. 2023Qwen Technical Reportcited by 3,344
  5. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  6. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  7. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  8. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  9. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  10. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  11. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  12. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  13. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  14. 2021Measuring Coding Challenge Competence With APPScited by 902
  15. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  16. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  17. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  18. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  19. 2022PAL: Program-aided Language Modelscited by 652
  20. 2019Improved Knowledge Distillation via Teacher Assistantcited by 573
  21. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 502
  22. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 498
  23. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 487
  24. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 452
  25. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 333
  26. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 300
  27. 2022Efficient Training of Language Models to Fill in the Middlecited by 255
  28. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 189
  29. 2023ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generationcited by 168
  30. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verificationcited by 166
  31. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoningcited by 156
  32. 2023PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedbackcited by 69
  33. 2024AST-T5: Structure-Aware Pretraining for Code Generation and Understandingcited by 33
  34. 2024Leveraging Print Debugging to Improve Code Generation in Large Language Modelscited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 7; 2026: 4.

14 papers.

Sort:

By area:

  1. 2025WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratchcited by 72
  2. 2024HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generationcited by 39
  3. 2025MathCanvas: Intrinsic Visual Chain-of-Thought for Multimodal Mathematical Reasoningcited by 35
  4. 2025ConvCodeWorld: Benchmarking Conversational Code Generation in Reproducible Feedback Environmentscited by 14
  5. 2024Multi-Programming Language Sandbox for LLMscited by 10
  6. 2025Enhancing LLM Code Generation: A Systematic Evaluation of Multi-Agent Collaboration and Runtime Debugging for Improved Accuracy, Reliability, and Latencycited by 9
  7. 2025Turning the Tide: Repository-based Code Reflectioncited by 5
  8. 2025ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation Groundingcited by 4
  9. 2025Alignment with Fill-In-the-Middle for Enhancing Code Generationcited by 4
  10. 2026MultiTalk: Scaling Full-Duplex Speech Models to Long, Multi-Party, Bilingual Conversationcited by 1
  11. 2024Investigating the Transferability of Code Repair for Low-Resource Programming Languagescited by 0
  12. 2026Selective Left-Shift: Turning Test-Time Compute and Difficulty-based Curation into Training Data for Low-Resource Code Generationcited by 0
2 more
  1. 2026Quantize with Confidence? An Empirical Study of Quantization for Code Generationcited by 0
  2. 2026What Shared Prefixes Hide: Trajectory Dropout for On-Policy Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Self-Reflective Generation at Test Time
  2. 2024Reasoning Through Execution: Unifying Process and Outcome Rewards for Code Generation
  3. 2024RethinkMCTS: Refining Erroneous Thoughts in Monte Carlo Tree Search for Code Generation
  4. 2025Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instruction
  5. 2023SelfEvolve: A Code Evolution Framework via Large Language Models
  6. 2024Repoformer: Selective Retrieval for Repository-Level Code Completion
  7. 2025RefleXGen:The unexamined code is not worth using
  8. 2025Chain of Execution Supervision Promotes General Reasoning in Large Language Models