# Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvement

arXiv: https://arxiv.org/abs/2411.00622 · HTML: https://mlsanity.com/paper/2411.00622/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 45 of its 68 references that are in the map.
- First posted on arXiv 1 Nov 2024.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 30 of 45)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [WizardCoder: Empowering Code Large Language Models with Evol-Instruct](https://mlsanity.com/paper/2306.08568/) (2023)
- [Executable Code Actions Elicit Better LLM Agents](https://mlsanity.com/paper/2402.01030/) (2024)
- [StarCoder 2 and The Stack v2: The Next Generation](https://mlsanity.com/paper/2402.19173/) (2024)
- [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020)
- [Agentless: Demystifying LLM-based Software Engineering Agents](https://mlsanity.com/paper/2407.01489/) (2024)
- [DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence](https://mlsanity.com/paper/2406.11931/) (2024)
- [CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges](https://mlsanity.com/paper/2401.07339/) (2024)
- [Magicoder: Empowering Code Generation with OSS-Instruct](https://mlsanity.com/paper/2312.02120/) (2023)
- [Large Language Model-Based Agents for Software Engineering: A Survey](https://mlsanity.com/paper/2409.02977/) (2024)
- [CodeGemma: Open Code Models Based on Gemma](https://mlsanity.com/paper/2406.11409/) (2024)
- [CodeR: Issue Resolving with Multi-Agent and Task Graphs](https://mlsanity.com/paper/2406.01304/) (2024)
- [At Which Training Stage Does Code Data Help LLMs Reasoning?](https://mlsanity.com/paper/2309.16298/) (2023)
- [MarsCode Agent: AI-native Automated Bug Fixing](https://mlsanity.com/paper/2409.00899/) (2024)
- [RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation](https://mlsanity.com/paper/2402.16667/) (2024)
- [PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedback](https://mlsanity.com/paper/2307.14936/) (2023)
- [CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases](https://mlsanity.com/paper/2408.03910/) (2024)
- [NExT: Teaching Large Language Models to Reason about Code Execution](https://mlsanity.com/paper/2404.14662/) (2024)
- [Alibaba LingmaAgent: Improving Automated Issue Resolution via Comprehensive Repository Exploration](https://mlsanity.com/paper/2406.01422/) (2024)
- [From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging](https://mlsanity.com/paper/2410.01215/) (2024)
- [CYCLE: Learning to Self-Refine the Code Generation](https://mlsanity.com/paper/2403.18746/) (2024)

## Built on by (most cited first, 30 of 61)

- [Training Software Engineering Agents and Verifiers with SWE-Gym](https://mlsanity.com/paper/2412.21139/) (2024)
- [SWE-smith: Scaling Data for Software Engineering Agents](https://mlsanity.com/paper/2504.21798/) (2025)
- [R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents](https://mlsanity.com/paper/2504.07164/) (2025)
- [WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratch](https://mlsanity.com/paper/2505.03733/) (2025)
- [Code as Agent Harness](https://mlsanity.com/paper/2605.18747/) (2026)
- [Context as a Tool: Context Management for Long-Horizon SWE-Agents](https://mlsanity.com/paper/2512.22087/) (2025)
- [Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learning](https://mlsanity.com/paper/2508.03501/) (2025)
- [BugPilot: Complex Bug Generation for Efficient Learning of SWE Skills](https://mlsanity.com/paper/2510.19898/) (2025)
- [SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuning](https://mlsanity.com/paper/2502.20127/) (2025)
- [Code Graph Model (CGM): A Graph-Integrated Large Language Model for Repository-Level Software Engineering Tasks](https://mlsanity.com/paper/2505.16901/) (2025)
- [Thinking Longer, Not Larger: Enhancing Software Engineering Agents via Scaling Test-Time Compute](https://mlsanity.com/paper/2503.23803/) (2025)
- [SWE-Dev: Building Software Engineering Agents with Training and Inference Scaling](https://mlsanity.com/paper/2506.07636/) (2025)
- [SWE-World: Building Software Engineering Agents in Docker-Free Environments](https://mlsanity.com/paper/2602.03419/) (2026)
- [PatchPilot: A Cost-Efficient Software Engineering Agent with Early Attempts on Formal Verification](https://mlsanity.com/paper/2502.02747/) (2025)
- [Issue Localization via LLM-Driven Iterative Code Graph Searching](https://mlsanity.com/paper/2503.22424/) (2025)
- [Cyber-Zero: Training Cybersecurity Agents without Runtime](https://mlsanity.com/paper/2508.00910/) (2025)
- [Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks](https://mlsanity.com/paper/2512.03262/) (2025)
- [The FM Agent](https://mlsanity.com/paper/2510.26144/) (2025)
- [Satori-SWE: Evolutionary Test-Time Scaling for Sample-Efficient Software Engineering](https://mlsanity.com/paper/2505.23604/) (2025)
- [Training Language Model Agents to Find Vulnerabilities with CTF-Dojo](https://mlsanity.com/paper/2508.18370/) (2025)
- [Beyond Final Code: A Process-Oriented Error Analysis of Software Development Agents in Real-World GitHub Scenarios](https://mlsanity.com/paper/2503.12374/) (2025)
- [Co-PatcheR: Collaborative Software Patching with Component(s)-specific Small Reasoning Models](https://mlsanity.com/paper/2505.18955/) (2025)
- [SEAlign: Alignment Training for Software Engineering Agent](https://mlsanity.com/paper/2503.18455/) (2025)
- [LLMs as Continuous Learners: Improving the Reproduction of Defective Code in Software Issues](https://mlsanity.com/paper/2411.13941/) (2024)
- [Guided Search Strategies in Non-Serializable Environments with Applications to Software Engineering Agents](https://mlsanity.com/paper/2505.13652/) (2025)
- [MemGovern: Enhancing Code Agents through Learning from Governed Human Experiences](https://mlsanity.com/paper/2601.06789/) (2026)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [Scaling Coding Agents via Atomic Skills](https://mlsanity.com/paper/2604.05013/) (2026)
- [Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application](https://mlsanity.com/paper/2606.12191/) (2026)
- [SWE-Factory: Your Automated Factory for Issue Resolution Training Data and Evaluation Benchmarks](https://mlsanity.com/paper/2506.10954/) (2025)

## Abstract (arXiv)

Recent advancements in LLM-based agents have led to significant progress in automatic software engineering, particularly in software maintenance and evolution. Despite these encouraging advances, current research faces two major challenges. First, SOTA performance primarily depends on closed-source models, which significantly limits the technology's accessibility, and potential for customization in diverse SE tasks. Second, these models are predominantly trained on static code data, lacking a deep understanding of the dynamic interactions, iterative problem-solving processes, and evolutionary characteristics inherent in software development. To address these challenges, our study adopts a software engineering perspective. We recognize that real-world software maintenance and evolution processes encompass not only static code data but also developers' thought processes, utilization of external tools, and the interaction between different functional personnel. Consequently, we introduce the Lingma SWE-GPT series, comprising Lingma SWE-GPT 7B and 72B. By learning from and simulating real-world code submission activities, Lingma SWE-GPT systematically incorporates the dynamic interactions and iterative problem-solving inherent in software development process, thereby achieving a more comprehensive understanding of software improvement processes. We conducted experimental evaluations using SWE-bench Verified benchmark. The results demonstrate that Lingma SWE-GPT 72B successfully resolves 30.20% of the GitHub issues, marking a significant improvement in automatic issue resolution (22.76% relative improvement compared to Llama 3.1 405B), approaching the performance of closed-source models (31.80% issues of GPT-4o resolved). Notably, Lingma SWE-GPT 7B resolves 18.20% of the issues, highlighting the potential for applying smaller models to ASE tasks.