mlsanityData to 7 Oct 2026

MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution

arXiv 2403.17927 ↗First posted 26 Mar 2024cs.SE cs.AIcited by 132

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

In software development, resolving the emergent issues within GitHub repositories is a complex challenge that involves not only the incorporation of new code but also the maintenance of existing code. Large Language Models (LLMs) have shown promise in code generation but face difficulties in resolving Github issues, particularly at the repository level. To overcome this challenge, we empirically study the reason why LLMs fail to resolve GitHub issues and analyze the major factors. Motivated by the empirical findings, we propose a novel LLM-based Multi-Agent framework for GitHub Issue reSolution, MAGIS, consisting of four agents customized for software evolution: Manager, Repository Custodian, Developer, and Quality Assurance Engineer agents. This framework leverages the collaboration of various agents in the planning and coding process to unlock the potential of LLMs to resolve GitHub issues. In experiments, we employ the SWE-bench benchmark to compare MAGIS with popular LLMs, including GPT-3.5, GPT-4, and Claude-2. MAGIS can resolve 13.94% GitHub issues, significantly outperforming the baselines. Specifically, MAGIS achieves an eight-fold increase in resolved ratio over the direct application of GPT-4, the advanced LLM.

Key facts. Cited by 132 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 25 of its 62 references that are in the map. First posted on arXiv 26 Mar 2024.

Builds on

25 of 62 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024RepairAgent: An Autonomous, LLM-Based Agent for Program Repair25 Mar

Other references · 24

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  7. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  8. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  9. 2023A Survey of Large Language Modelscited by 2,537
  10. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  11. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  12. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
12 more
  1. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  2. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 329
  3. 2024Data Interpreter: An LLM Agent For Data Sciencecited by 222
  4. 2022Practical Program Repair in the Era of Large Pre-trained Language Modelscited by 133
  5. 2023Thread of Thought Unraveling Chaotic Contextscited by 71
  6. 2023Towards an Understanding of Large Language Models in Software Engineering Taskscited by 67
  7. 2023Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Codecited by 62
  8. 2023RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repaircited by 43
  9. 2023Developer-Intent Driven Code Comment Generationcited by 16
  10. 2023ITER: Iterative Neural Repair for Multi-Location Patchescited by 16
  11. 2023PTP: Boosting Stability and Performance of Prompt Tuning with Perturbation-Based Regularizercited by 15
  12. 2024KADEL: Knowledge-Aware Denoising Learning for Commit Message Generationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 29; 2025: 50; 2026: 53.

132 papers.

Sort:

By area:

  1. 2024AutoCodeRover: Autonomous Program Improvementcited by 272
  2. 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 223
  3. 2024Large Language Model-Based Agents for Software Engineering: A Surveycited by 173
  4. 2024LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Aheadcited by 155
  5. 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 145
  6. 2024SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agentscited by 129
  7. 2024LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-trainingcited by 122
  8. 2024Teams of LLM Agents can Exploit Zero-Day Vulnerabilitiescited by 93
  9. 2024CodeR: Issue Resolving with Multi-Agent and Task Graphscited by 89
  10. 2024From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Futurecited by 85
  11. 2025Understanding Software Engineering Agents: A Study of Thought-Action-Result Trajectoriescited by 75
  12. 2025Latent Collaboration in Multi-Agent Systemscited by 67
38 more
  1. 2025MARFT: Multi-Agent Reinforcement Fine-Tuningcited by 51
  2. 2024You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projectscited by 45
  3. 2024Human-In-the-Loop Software Development Agentscited by 42
  4. 2025General Agentic Memory Via Deep Researchcited by 42
  5. 2024Can LLMs Replace Manual Annotation of Software Engineering Artifacts?cited by 40
  6. 2025Collab-Overcooked: Benchmarking and Evaluating Large Language Models as Collaborative Agentscited by 39
  7. 2025MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaborationcited by 39
  8. 2024TDD-Bench Verified: Can LLMs Generate Tests for Issues Before They Get Resolved?cited by 32
  9. 2025Stronger-MAS: Multi-Agent Reinforcement Learning for Collaborative LLMscited by 25
  10. 2025Process-Centric Analysis of Agentic Software Systemscited by 24
  11. 2024An Empirical Study on LLM-based Agents for Automated Bug Fixingcited by 23
  12. 2025PatchPilot: A Cost-Efficient Software Engineering Agent with Early Attempts on Formal Verificationcited by 22
  13. 2025Heterogeneous Swarms: Jointly Optimizing Model Roles and Weights for Multi-LLM Systemscited by 22
  14. 2025Large Language Model-based Data Science Agent: A Surveycited by 22
  15. 2025Emergent Coordination in Multi-Agent Language Modelscited by 22
  16. 2024Beyond pip install: Evaluating LLM Agents for the Automated Installation of Python Projectscited by 21
  17. 2026CooperBench: Why Coding Agents Cannot be Your Teammates Yetcited by 20
  18. 2026Building Effective AI Coding Agents for the Terminal: Scaffolding, Harness, Context Engineering, and Lessons Learnedcited by 18
  19. 2026Uncovering Security Threats and Architecting Defenses in Autonomous Agents: A Case Study of OpenClawcited by 18
  20. 2024AIGS: Generating Science from AI-Powered Automated Falsificationcited by 17
  21. 2025SWE-Effi: Re-Evaluating Software AI Agent System Effectiveness Under Resource Constraintscited by 17
  22. 2024MdEval: Massively Multilingual Code Debuggingcited by 16
  23. 2025A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic Systemcited by 15
  24. 2025MIND: A Multi-agent Framework for Zero-shot Harmful Meme Detectioncited by 14
  25. 2025An Empirical Study of Agent Developer Practices in AI Agent Frameworkscited by 14
  26. 2024Top General Performance = Top Domain Performance? DomainCodeBench: A Multi-domain Code Generation Benchmarkcited by 13
  27. 2025Controlling Performance and Budget of a Centralized Multi-agent LLM System with Reinforcement Learningcited by 13
  28. 2024AutoPrep: Natural Language Question-Aware Data Preparation with a Multi-Agent Frameworkcited by 12
  29. 2025Towards Transparent and Incentive-Compatible Collaboration in Decentralized LLM Multi-Agent Systems: A Blockchain-Driven Approachcited by 11
  30. 2026Agent Primitives: Reusable Latent Building Blocks for Multi-Agent Systemscited by 11
  31. 2026Failure as a Process: An Anatomy of CLI Coding Agent Trajectoriescited by 11
  32. 2024Scattered Forest Search: Smarter Code Space Exploration with LLMscited by 10
  33. 2025Game Theory Meets LLM and Agentic AI: Reimagining Cybersecurity for the Age of Intelligent Threatscited by 10
  34. 2025Aime: Towards Fully-Autonomous Multi-Agent Frameworkcited by 9
  35. 2026AgentStepper: Interactive Debugging of Software Development Agentscited by 8
  36. 2026Budget-Constrained Agentic Large Language Models: Intention-Based Planning for Costly Tool Usecited by 8
  37. 2026The Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systemscited by 8
  38. 2025Optimizing LLM-Based Multi-Agent System with Textual Feedback: A Case Study on Software Developmentcited by 7

50 most cited of 132 All 132 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026XAgent: eXecution-guided Agentic AI for Effective Localization and Resolution of GitHub Issues
  2. 2026SWE-Adept: An LLM-Based Agentic Framework for Deep Codebase Analysis and Structured Issue Resolution
  3. 2026Phoenix: Safe GitHub Issue Resolution via Multi-Agent LLMs
  4. 2026CodeTeam: An LLM-Powered Multi-Agent Framework for Repository-Level Code Generation
  5. 2024Evaluating Software Development Agents: Patch Patterns, Code Quality, and Issue Complexity in Real-World GitHub Scenarios
  6. 2025On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub
  7. 2026BeyondSWE: Can Current Code Agent Survive Beyond Single-Repo Bug Fixing?
  8. 2026REAgent: Requirement-Driven LLM Agents for Software Issue Resolution