mlsanityData to 14 Oct 2026

PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedback

arXiv 2307.14936 ↗First posted 27 Jul 2023cs.CL cs.AI cs.LGcited by 69

Not on the map. Why?

Abstract (arXiv)

Large Language Models for Code (Code LLM) are flourishing. New and powerful models are released on a weekly basis, demonstrating remarkable performance on the code generation task. Various approaches have been proposed to boost the code generation performance of pre-trained Code LLMs, such as supervised fine-tuning, instruction tuning, reinforcement learning, etc. In this paper, we propose a novel RRTF (Rank Responses to align Test&Teacher Feedback) framework, which can effectively and efficiently boost pre-trained large language models for code generation. Under this framework, we present PanGu-Coder2, which achieves 62.20% pass@1 on the OpenAI HumanEval benchmark. Furthermore, through an extensive evaluation on CoderEval and LeetCode benchmarks, we show that PanGu-Coder2 consistently outperforms all previous Code LLMs.

Key facts. Cited by 69 papers in the map (data through 14 Oct 2026). Builds on 39 of its 42 references that are in the map. First posted on arXiv 27 Jul 2023.

Builds on

39 of 42 references.

References · 39

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  5. 2021Program Synthesis with Large Language Modelscited by 3,637
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  7. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  8. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  9. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  10. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  11. 2023LIMA: Less Is More for Alignmentcited by 1,100
  12. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
27 more
  1. 2023StarCoder: may the source be with you!cited by 799
  2. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  3. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  4. 2023Instruction Tuning with GPT-4cited by 686
  5. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  6. 2023Textbooks Are All You Needcited by 569
  7. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 498
  8. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 463
  9. 2022CodeT: Code Generation with Generated Testscited by 384
  10. 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 367
  11. 2022A Systematic Evaluation of Large Language Models of Codecited by 363
  12. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  13. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 333
  14. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 300
  15. 2022Efficient Training of Language Models to Fill in the Middlecited by 255
  16. 2023SantaCoder: don't reach for the stars!cited by 170
  17. 2022Large Language Models Meet NL2Code: A Surveycited by 138
  18. 2022ReACC: A Retrieval-Augmented Code Completion Frameworkcited by 119
  19. 2023RLTF: Reinforcement Learning from Unit Test Feedbackcited by 114
  20. 2023Execution-based Code Generation using Deep Reinforcement Learningcited by 99
  21. 2022CERT: Continual Pre-Training on Sketches for Library-Oriented Code Generationcited by 87
  22. 2022When Language Model Meets Private Librarycited by 54
  23. 2022PanGu-Coder: Program Synthesis with Function-Level Language Modelingcited by 48
  24. 2022Training and Evaluating a Jupyter Notebook Data Science Assistantcited by 31
  25. 2022ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languagescited by 29
  26. 2023CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Modelscited by 28
  27. 2023Meet in the Middle: A New Pre-training Paradigmcited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 20; 2024: 31; 2025: 13; 2026: 5.

69 papers.

Sort:

By area:

  1. 2024A Survey on Large Language Models for Code Generationcited by 658
  2. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 300
  3. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 189
  4. 2023ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generationcited by 168
  5. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 160
  6. 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 136
  7. 2023Can ChatGPT replace StackOverflow? A Study on Robustness and Reliability of Large Language Model Code Generationcited by 73
  8. 2023Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Codecited by 62
  9. 2024Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvementcited by 61
  10. 2024Agents in Software Engineering: Survey, Landscape, and Visioncited by 60
  11. 2023LLM-Powered Hierarchical Language Agent for Real-time Human-AI Coordinationcited by 48
  12. 2024CodeJudge: Evaluating Code Generation with Large Language Modelscited by 44
38 more
  1. 2023WaveCoder: Widespread And Versatile Enhancement For Code Large Language Models By Instruction Tuningcited by 42
  2. 2024Dataflow-Guided Retrieval Augmentation for Repository-Level Code Completioncited by 40
  3. 2024Benchmarking Agentic Workflow Generationcited by 37
  4. 2025LessLeak-Bench: A First Investigation of Data Leakage in LLMs Across 83 Software Engineering Benchmarkscited by 37
  5. 2023MFTCoder: Boosting Code LLMs with Multitask Fine-Tuningcited by 35
  6. 2023Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answeringcited by 29
  7. 2024ToolPlanner: A Tool Augmented LLM for Multi Granularity Instructions with Path Planning and Feedbackcited by 29
  8. 2024CodeS: Natural Language to Code Repository via Multi-Layer Sketchcited by 28
  9. 2024LLM-Empowered State Representation for Reinforcement Learningcited by 28
  10. 2024Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generationcited by 27
  11. 2023LORD: Low Rank Decomposition Of Monolingual Code LLMs For One-Shot Compressioncited by 26
  12. 2023BioCoder: A Benchmark for Bioinformatics Code Generation with Large Language Modelscited by 25
  13. 2023Competition-Level Problems are Effective LLM Evaluatorscited by 25
  14. 2024Unlocking Reasoning Potential in Large Langauge Models by Scaling Code-form Planningcited by 25
  15. 2024Kolb-Based Experiential Learning for Generalist Agents with Human-Level Kaggle Data Science Performancecited by 25
  16. 2024Astraios: Parameter-Efficient Instruction Tuning Code Large Language Modelscited by 23
  17. 2023LLM for SoC Security: A Paradigm Shiftcited by 22
  18. 2023Tool-Augmented Reward Modelingcited by 20
  19. 2024LocMoE: A Low-Overhead MoE for Large Language Model Trainingcited by 19
  20. 2023INTERVENOR: Prompting the Coding Ability of Large Language Models with the Interactive Chain of Repaircited by 18
  21. 2024Towards a Unified View of Preference Learning for Large Language Models: A Surveycited by 17
  22. 2023Why Can Large Language Models Generate Correct Chain-of-Thoughts?cited by 16
  23. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 15
  24. 2023CodeFuse-13B: A Pretrained Multi-lingual Code Large Language Modelcited by 14
  25. 2024ReflectionCoder: Learning from Reflection Sequence for Enhanced One-off Code Generationcited by 14
  26. 2024Top General Performance = Top Domain Performance? DomainCodeBench: A Multi-domain Code Generation Benchmarkcited by 13
  27. 2025ReachAgent: Enhancing Mobile Agent via Page Reaching and Operationcited by 13
  28. 2025a1: Steep Test-time Scaling Law via Environment Augmented Generationcited by 13
  29. 2024Learning to Plan for Language Modeling from Unlabeled Datacited by 12
  30. 2023Data Race Detection Using Large Language Modelscited by 11
  31. 2024DolphCoder: Echo-Locating Code Large Language Models with Diverse and Multi-Objective Instruction Tuningcited by 11
  32. 2024Preference-Guided Refactored Tuning for Retrieval Augmented Code Generationcited by 10
  33. 2024Coffee-Gym: An Environment for Evaluating and Improving Natural Language Feedback on Erroneous Codecited by 9
  34. 2026Code-A1: Adversarial Evolving of Code LLM and Test LLM via Reinforcement Learningcited by 8
  35. 2023Can Programming Languages Boost Each Other via Instruction Tuning?cited by 7
  36. 2024JumpCoder: Go Beyond Autoregressive Coder via Online Modificationcited by 7
  37. 2024Exploring the Potential of Large Language Models for Improving Digital Forensic Investigation Efficiencycited by 7
  38. 2025The Art of Repair: Optimizing Iterative Program Repair with Instruction-Tuned Modelscited by 7

50 most cited of 69 All 69 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025CodeBoost: Boosting Code LLMs by Squeezing Knowledge from Code Snippets with RL
  2. 2024StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback
  3. 2025CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks
  4. 2025RefineCoder: Iterative Improving of Large Language Models via Adaptive Critique Refinement for Code Generation
  5. 2025OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critique
  6. 2023Improving Code Generation by Training with Natural Language Feedback
  7. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning
  8. 2026CodeEval: A pedagogical approach for targeted evaluation of code-trained Large Language Models