mlsanityData to 7 Oct 2026

MapCoder: Multi-Agent Code Generation for Competitive Problem Solving

arXiv 2405.11403 ↗First posted 18 May 2024cs.CL cs.AIcited by 177

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Code synthesis, which requires a deep understanding of complex natural language problem descriptions, generation of code instructions for complex algorithms and data structures, and the successful execution of comprehensive unit tests, presents a significant challenge. While large language models (LLMs) demonstrate impressive proficiency in natural language processing, their performance in code generation tasks remains limited. In this paper, we introduce a new approach to code generation tasks leveraging multi-agent prompting that uniquely replicates the full cycle of program synthesis as observed in human developers. Our framework, MapCoder, consists of four LLM agents specifically designed to emulate the stages of this cycle: recalling relevant examples, planning, code generation, and debugging. After conducting thorough experiments, with multiple LLM ablations and analyses across eight challenging competitive problem-solving and program synthesis benchmarks, MapCoder showcases remarkable code generation capabilities, achieving new state-of-the-art results (pass@1) on HumanEval (93.9%), MBPP (83.1%), APPS (22.0%), CodeContests (28.5%), and xCodeEval (45.3%). Moreover, our method consistently delivers superior performance across various programming languages and varying problem difficulties. We open-source our framework at https://github.com/Md-Ashraful-Pramanik/MapCoder.

Key facts. Cited by 177 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 30 of its 48 references that are in the map. First posted on arXiv 18 May 2024.

Builds on

30 of 48 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023MoTCoder: Elevating Large Language Models with Modular of Thought for Challenging Programming Tasks26 Dec
  2. 2023Teaching Large Language Models to Self-Debug11 Apr
  3. 2022Competition-Level Code Generation with AlphaCode8 Feb

Other references · 27

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  4. 2021Program Synthesis with Large Language Modelscited by 3,509
  5. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  6. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  7. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  8. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  9. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  10. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  11. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  12. 2022CodeT: Code Generation with Generated Testscited by 369
15 more
  1. 2021Unified Pre-training for Program Understanding and Generationcited by 361
  2. 2023Self-collaboration Code Generation via ChatGPTcited by 255
  3. 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 180
  4. 2023SantaCoder: don't reach for the stars!cited by 169
  5. 2021Retrieval Augmented Code Generation and Summarizationcited by 148
  6. 2020Task-Oriented Dialogue as Dataflow Synthesiscited by 122
  7. 2023Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Datacited by 119
  8. 2023Large Language Models as Analogical Reasonerscited by 107
  9. 2024Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineeringcited by 100
  10. 2017Abstract Syntax Networks for Code Generation and Semantic Parsingcited by 92
  11. 2017Neural Map: Structured Memory for Deep Reinforcement Learningcited by 88
  12. 2023Learning to Filter Context for Retrieval-Augmented Generationcited by 82
  13. 2023CodeScore: Evaluating Code Generation by Learning Code Executioncited by 66
  14. 2018Building Language Models for Text with Named Entitiescited by 15
  15. 2022Retrieval Enhanced Data Augmentation for Question Answering on Privacy Policiescited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 23; 2025: 90; 2026: 64.

177 papers.

Sort:

By area:

  1. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  2. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  3. 2024Large Language Model-Based Agents for Software Engineering: A Surveycited by 173
  4. 2025MultiAgentBench: Evaluating the Collaboration and Competition of LLM agentscited by 169
  5. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 146
  6. 2025Memory Injection Attacks on LLM Agents via Query-Only Interactioncited by 127
  7. 2024Teams of LLM Agents can Exploit Zero-Day Vulnerabilitiescited by 93
  8. 2024From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Futurecited by 85
  9. 2024A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendationscited by 84
  10. 2024Self-Evolving Multi-Agent Collaboration Networks for Software Developmentcited by 73
  11. 2024HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scalecited by 61
  12. 2025LLM Agents for Education: Advances and Applicationscited by 59
38 more
  1. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  2. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 45
  3. 2026SkillOS: Learning Skill Curation for Self-Evolving Agentscited by 45
  4. 2025EvoAgentX: An Automated Framework for Evolving Agentic Workflowscited by 43
  5. 2025X-MAS: Towards Building Multi-Agent Systems with Heterogeneous LLMscited by 33
  6. 2024CodeTree: Agent-guided Tree Search for Code Generation with Large Language Modelscited by 32
  7. 2025CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debuggingcited by 31
  8. 2024What Makes Large Language Models Reason in (Multi-Turn) Code Generation?cited by 29
  9. 2025LMR-BENCH: Evaluating LLM Agent's Ability on Reproducing Language Modeling Researchcited by 29
  10. 2025Multi-Turn Code Generation Through Single-Step Rewardscited by 27
  11. 2024DataNarrative: Automated Data-Driven Storytelling with Visualizations and Textscited by 25
  12. 2024Agents4PLC: Automating Closed-loop PLC Code Generation and Verification in Industrial Control Systems using LLM-based Agentscited by 25
  13. 2025EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Codecited by 24
  14. 2025QualityFlow: An Agentic Workflow for Program Synthesis Controlled by LLM Quality Checkscited by 22
  15. 2025OptimAI: Optimization from Natural Language Using LLM-Powered AI Agentscited by 22
  16. 2025Large Language Model-based Data Science Agent: A Surveycited by 22
  17. 2025nvAgent: Automated Data Visualization from Natural Language via Collaborative Agent Workflowcited by 21
  18. 2026SkillJect: Effectively Automating Skill-Based Prompt Injection for Skill-Enabled Agentscited by 20
  19. 2025SEW: Self-Evolving Agentic Workflows for Automated Code Generationcited by 18
  20. 2025LL3M: Large Language 3D Modelerscited by 18
  21. 2025CodeFlowBench: A Multi-turn, Iterative Benchmark for Complex Code Generationcited by 16
  22. 2026Silo-Bench: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systemscited by 16
  23. 2025CompileAgent: Automated Real-World Repo-Level Compilation with Tool-Integrated LLM-based Agent Systemcited by 15
  24. 2025P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmarkcited by 15
  25. 2025MASLab: A Unified and Comprehensive Codebase for LLM-based Multi-Agent Systemscited by 14
  26. 2025DeepCode: Open Agentic Codingcited by 14
  27. 2025MaintainCoder: Maintainable Code Generation Under Dynamic Requirementscited by 13
  28. 2025SWE-Dev: Evaluating and Training Autonomous Feature-Driven Software Developmentcited by 13
  29. 2025AgentMaster: A Multi-Agent Conversational Framework Using A2A and MCP Protocols for Multimodal Information Retrieval and Analysiscited by 13
  30. 2024RGD: Multi-LLM Based Agent Debugger via Refinement and Generation Guidancecited by 12
  31. 2024Tree-of-Code: A Tree-Structured Exploring Framework for End-to-End Code Generation and Execution in Complex Task Handlingcited by 11
  32. 2025SOP-Agent: Empower General Purpose AI Agent with Domain-Specific SOPscited by 11
  33. 2025The Hidden Strength of Disagreement: Unraveling the Consensus-Diversity Tradeoff in Adaptive Multi-Agent Systemscited by 11
  34. 2025BannerAgency: Advertising Banner Design with Multimodal LLM Agentscited by 11
  35. 2025FlowerTune: A Cross-Domain Benchmark for Federated Fine-Tuning of Large Language Modelscited by 11
  36. 2026MemPO: Self-Memory Policy Optimization for Long-Horizon Agentscited by 11
  37. 2024Scattered Forest Search: Smarter Code Space Exploration with LLMscited by 10
  38. 2025LLM4EFFI: Leveraging Large Language Models to Enhance Code Efficiency and Correctnesscited by 10

50 most cited of 177 All 177 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation
  2. 2025CodeCoR: An LLM-Based Self-Reflective Multi-Agent Framework for Code Generation
  3. 2025Guided Code Generation with LLMs: A Multi-Agent Framework for Complex Code Tasks
  4. 2025CodeVisionary: An Agent-based Framework for Evaluating Large Language Models in Code Generation
  5. 2026No Test Cases, No Problem: Distillation-Driven Code Generation for Scientific Workflows
  6. 2025Can Code Language Models Learn Clarification-Seeking Behaviors?
  7. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence
  8. 2024Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimization