# CompileAgent: Automated Real-World Repo-Level Compilation with Tool-Integrated LLM-based Agent System

arXiv: https://arxiv.org/abs/2505.04254 · HTML: https://mlsanity.com/paper/2505.04254/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 22 of its 37 references that are in the map.
- First posted on arXiv 7 May 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 22 of 22)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https://mlsanity.com/paper/2405.04434/) (2024)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning](https://mlsanity.com/paper/2205.09712/) (2022)
- [ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs](https://mlsanity.com/paper/2309.13007/) (2023)
- [CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges](https://mlsanity.com/paper/2401.07339/) (2024)
- [RepairAgent: An Autonomous, LLM-Based Agent for Program Repair](https://mlsanity.com/paper/2403.17134/) (2024)
- [MapCoder: Multi-Agent Code Generation for Competitive Problem Solving](https://mlsanity.com/paper/2405.11403/) (2024)
- [Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?](https://mlsanity.com/paper/2402.18272/) (2024)
- [OpenAgents: An Open Platform for Language Agents in the Wild](https://mlsanity.com/paper/2310.10634/) (2023)
- [Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents](https://mlsanity.com/paper/2402.09205/) (2024)
- [MarsCode Agent: AI-native Automated Bug Fixing](https://mlsanity.com/paper/2409.00899/) (2024)
- [Towards End-to-End Embodied Decision Making via Multi-modal Large Language Model: Explorations with GPT4-Vision and Beyond](https://mlsanity.com/paper/2310.02071/) (2023)
- [PentestAgent: Incorporating LLM Agents to Automated Penetration Testing](https://mlsanity.com/paper/2411.05185/) (2024)
- [Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement](https://mlsanity.com/paper/2305.14497/) (2023)
- [QueryAgent: A Reliable and Efficient Reasoning Framework with Environmental Feedback-based Self-Correction](https://mlsanity.com/paper/2403.11886/) (2024)
- [CP-BCS: Binary Code Summarization Guided by Control Flow Graph and Pseudo Code](https://mlsanity.com/paper/2310.16853/) (2023)
- [BinaryAI: Binary Software Composition Analysis via Intelligent Binary Source Code Matching](https://mlsanity.com/paper/2401.11161/) (2024)

## Built on by (most cited first, 15 of 15)

- [VIGIL: Defending LLM Agents Against Tool Stream Injection via Verify-Before-Commit](https://mlsanity.com/paper/2601.05755/) (2026)
- [AgentMaster: A Multi-Agent Conversational Framework Using A2A and MCP Protocols for Multimodal Information Retrieval and Analysis](https://mlsanity.com/paper/2507.21105/) (2025)
- [TrajAudit: Automated Failure Diagnosis for Agentic Coding Systems](https://mlsanity.com/paper/2605.26563/) (2026)
- [Code2MCP: Transforming Code Repositories into MCP Services](https://mlsanity.com/paper/2509.05941/) (2025)
- [Agentic Harness for Real-World Compilers](https://mlsanity.com/paper/2603.20075/) (2026)
- [BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source Software](https://mlsanity.com/paper/2509.25248/) (2025)
- [Large Language Models for Agentic NetOps and AIOps: Architectures, Evaluation, and Safety](https://mlsanity.com/paper/2605.12729/) (2026)
- [Towards Direct Evaluation of Harness Optimizers via Priority Ranking](https://mlsanity.com/paper/2605.22505/) (2026)
- [LLMs as Packagers of HPC Software](https://mlsanity.com/paper/2511.05626/) (2025)
- [Cross-modal Retrieval Models for Stripped Binary Analysis](https://mlsanity.com/paper/2512.10393/) (2025)
- [From LLMs to Agents in Programming: The Impact of Providing an LLM with a Compiler](https://mlsanity.com/paper/2601.12146/) (2026)
- [Challenges and Future Directions in Agentic Reverse Engineering Systems](https://mlsanity.com/paper/2604.14317/) (2026)
- [AutoPass: Evidence-Guided LLM Agents for Compiler Performance Tuning](https://mlsanity.com/paper/2606.20373/) (2026)
- [The New Compiler Stack: A Survey on the Synergy of LLMs and Compilers](https://mlsanity.com/paper/2601.02045/) (2026)
- [Only Say What You Know: Calibration-Aware Generation for Long-Form Factuality](https://mlsanity.com/paper/2605.01749/) (2026)

## Abstract (arXiv)

With open-source projects growing in size and complexity, manual compilation becomes tedious and error-prone, highlighting the need for automation to improve efficiency and accuracy. However, the complexity of compilation instruction search and error resolution makes automatic compilation challenging. Inspired by the success of LLM-based agents in various fields, we propose CompileAgent, the first LLM-based agent framework dedicated to repo-level compilation. CompileAgent integrates five tools and a flow-based agent strategy, enabling interaction with software artifacts for compilation instruction search and error resolution. To measure the effectiveness of our method, we design a public repo-level benchmark CompileAgentBench, and we also design two baselines for comparison by combining two compilation-friendly schemes. The performance on this benchmark shows that our method significantly improves the compilation success rate, ranging from 10% to 71%. Meanwhile, we evaluate the performance of CompileAgent under different agent strategies and verify the effectiveness of the flow-based strategy. Additionally, we emphasize the scalability of CompileAgent, further expanding its application prospects.