# CodeCoR: An LLM-Based Self-Reflective Multi-Agent Framework for Code Generation

arXiv: https://arxiv.org/abs/2501.07811 · HTML: https://mlsanity.com/paper/2501.07811/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 29 of its 49 references that are in the map.
- First posted on arXiv 14 Jan 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 29 of 29)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [Reasoning with Language Model is Planning with World Model](https://mlsanity.com/paper/2305.14992/) (2023)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct](https://mlsanity.com/paper/2308.09583/) (2023)
- [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022)
- [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020)
- [CodeT: Code Generation with Generated Tests](https://mlsanity.com/paper/2207.10397/) (2022)
- [CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges](https://mlsanity.com/paper/2401.07339/) (2024)
- [Generating Sequences by Learning to Self-Correct](https://mlsanity.com/paper/2211.00053/) (2022)
- [Self-collaboration Code Generation via ChatGPT](https://mlsanity.com/paper/2304.07590/) (2023)
- [Is Self-Repair a Silver Bullet for Code Generation?](https://mlsanity.com/paper/2306.09896/) (2023)
- [Structured Chain-of-Thought Prompting for Code Generation](https://mlsanity.com/paper/2305.06599/) (2023)
- [Self-planning Code Generation with Large Language Models](https://mlsanity.com/paper/2303.06689/) (2023)
- [Self-Edit: Fault-Aware Code Editor for Code Generation](https://mlsanity.com/paper/2305.04087/) (2023)
- [Leveraging Automated Unit Tests for Unsupervised Code Translation](https://mlsanity.com/paper/2110.06773/) (2021)
- [DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories](https://mlsanity.com/paper/2405.19856/) (2024)
- [CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules](https://mlsanity.com/paper/2310.08992/) (2023)
- [CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models](https://mlsanity.com/paper/2411.04329/) (2024)

## Built on by (most cited first, 30 of 34)

- [Code as Agent Harness](https://mlsanity.com/paper/2605.18747/) (2026)
- [Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI](https://mlsanity.com/paper/2505.19443/) (2025)
- [The FM Agent](https://mlsanity.com/paper/2510.26144/) (2025)
- [AutoP2C: An LLM-Based Agent Framework for Code Repository Generation from Multimodal Content in Academic Papers](https://mlsanity.com/paper/2504.20115/) (2025)
- [Position: Foundation Models for Tabular Data within Systemic Contexts Need Grounding](https://mlsanity.com/paper/2505.19825/) (2025)
- [EvoMAS: Evolutionary Generation of Multi-Agent Systems](https://mlsanity.com/paper/2602.06511/) (2026)
- [NL-Debugging: Exploiting Natural Language as an Intermediate Representation for Code Debugging](https://mlsanity.com/paper/2505.15356/) (2025)
- [Cross-Model LLM Code Review: Should you use Claude to review Codex or vice versa?](https://mlsanity.com/paper/2607.21656/) (2026)
- [MapCoder-Lite: Distilling Multi-Agent Coding into a Single Small LLM](https://mlsanity.com/paper/2509.17489/) (2025)
- [Designing LLM-based Multi-Agent Systems for Software Engineering Tasks: Quality Attributes, Design Patterns and Rationale](https://mlsanity.com/paper/2511.08475/) (2025)
- [LLM-Based Multi-Agent Systems for Code Generation: A Multi-Vocal Literature Review](https://mlsanity.com/paper/2604.16321/) (2026)
- [PropGuard: Safeguarding LLM-MAS via Propagation-Aware Exploration and Remediation](https://mlsanity.com/paper/2605.16346/) (2026)
- [MoT: Modularization-of-Thought Prompting for Effective Code Generation](https://mlsanity.com/paper/2503.12483/) (2025)
- [Beyond Language Barriers: Multi-Agent Coordination for Multi-Language Code Generation](https://mlsanity.com/paper/2509.19918/) (2025)
- [Rollout-Training Co-Design for Efficient LLM-Based Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2602.09578/) (2026)
- [Test Amplification for REST APIs via Single and Multi-Agent LLM Systems](https://mlsanity.com/paper/2504.08113/) (2025)
- [MemoCoder: Automated Function Synthesis using LLM-Supported Agents](https://mlsanity.com/paper/2507.18812/) (2025)
- [From LLMs to Agents: A Comparative Evaluation of LLMs and LLM-based Agents in Security Patch Detection](https://mlsanity.com/paper/2511.08060/) (2025)
- [AI-for-Science Low-code Platform with Bayesian Adversarial Multi-Agent Framework](https://mlsanity.com/paper/2603.03233/) (2026)
- [Persistent Cross-Attempt State Optimization for Repository-Level Code Generation](https://mlsanity.com/paper/2604.03632/) (2026)
- [ARIADNE: Agentic Reward-Informed Adaptive Decision Exploration via Blackboard-Driven MCTS for Competitive Program Generation](https://mlsanity.com/paper/2605.02431/) (2026)
- [FASE: Fast Adaptive Semantic Entropy for Code Quality](https://mlsanity.com/paper/2606.09800/) (2026)
- [AutoPass: Evidence-Guided LLM Agents for Compiler Performance Tuning](https://mlsanity.com/paper/2606.20373/) (2026)
- [LOCA: Logical Chain Augmentation for Scientific Corpus Cleaning](https://mlsanity.com/paper/2510.01249/) (2025)
- [Knowledge-Guided Multi-Agent Framework for Application-Level Software Code Generation](https://mlsanity.com/paper/2510.19868/) (2025)
- [SwiftSolve: A Self-Iterative, Complexity-Aware Multi-Agent Framework for Competitive Programming](https://mlsanity.com/paper/2510.22626/) (2025)
- [Lifecycle-Aware code generation: Leveraging Software Engineering Phases in LLMs](https://mlsanity.com/paper/2510.24019/) (2025)
- [Agentic LLMs for REST API Test Amplification: A Comparative Study Across Cloud Applications](https://mlsanity.com/paper/2510.27417/) (2025)
- [AutoB2G: Agentic Simulation and Reinforcement Learning for Spatio-Temporal Grid-Interactive Building Control](https://mlsanity.com/paper/2603.26005/) (2026)
- [You Don't Need Public Tests to Generate Correct Code](https://mlsanity.com/paper/2604.21598/) (2026)

## Abstract (arXiv)

Code generation aims to produce code that fulfills requirements written in natural language automatically. Large Language Models (LLMs) like ChatGPT have demonstrated promising effectiveness in this area. Nonetheless, the LLMs often fail to ensure the syntactic and semantic correctness of the generated code. Recently, researchers proposed multi-agent frameworks that guide LLMs with different prompts to analyze programming tasks, generate code, and perform testing in a sequential workflow. However, the performance of the workflow is not robust as code generation depends on the performance of each agent. To address this challenge, we propose CodeCoR, a multi-agent framework that prunes intermediate outputs at different stages to reduce error propagation in sequential code generation workflows. Specifically, for a given task description, four agents in CodeCoR generate prompts, code, test cases, and repair advice, respectively. Each agent generates more than one output and prunes away the low-quality ones. The generated code is tested in the local environment: the code that fails to pass the generated test cases is sent to the repair agent, and the coding agent re-generates the code based on repair advice. Finally, the code that passes the highest number of generated test cases is returned to the users. Our experiments on four widely used datasets, HumanEval, HumanEval-ET, MBPP, and MBPP-ET, demonstrate that CodeCoR outperforms existing baselines (e.g., CodeCoT and MapCoder), achieving an average Pass@1 score of 77.13%.