mlsanityData to 7 Oct 2026

SetupX: Can LLM Agents Learn from Past Failures in Functionality-Correct Code Repository Setup?

arXiv 2605.26186 ↗First posted 25 May 2026cs.SE cs.AI cs.CLcited by 1

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Functionality-correct repository setup aims to configure execution environments (e.g., dependencies, build scripts) to successfully execute a repository's documented features. It presents significant challenges due to diverse, repository-specific failures, including dependency incompatibilities, missing toolchains, incomplete installations, and verification-strategy mismatches. Existing LLM agents struggle to robustly resolve these issues, specifically failing to support (1) cross-repository experience transfer, (2) multi-step trial-and-repair under non-invertible state changes, and (3) robust verification of setup outcomes to distinguish setup-induced failures from repository bugs. To address this, we introduce SetupX, an experiential learning-based setup framework. First, we construct a Self-Evolving Experience Representation (XPU), a dual-modality knowledge unit encoding setup signals, textual guidance, executable actions to dynamically transfer verified environment fixes to unseen repositories. Second, we employ Experience-Augmented Speculative Execution backed by a LIFO Docker snapshot stack, enabling the agent to proactively trial fixes and safely roll back to known-good states. Third, we introduce a Prosecutor-Judge Verification Protocol that separates evidence collection from final judgment, enabling more reliable setup verification beyond superficial build-time metrics. Evaluation results on carefully-crafted benchmarks show SetupX achieves highest performance (e.g., 92% pass rate) and outperforms the strongest baseline by over 19%. Crucially, SetupX excels in complex multi-repository setup requiring coordinating multiple interconnected services across different containers. The code repository is available at https://github.com/OpenDataBox/SetupX.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 20 of its 23 references that are in the map. First posted on arXiv 25 May 2026.

Builds on

20 of 23 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projects13 Dec
  2. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agents23 Jul
  3. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May

Other references · 17

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  4. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  5. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  6. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  7. 2024AutoCodeRover: Autonomous Program Improvementcited by 272
  8. 2025EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecyclecited by 129
  9. 2025Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solvingcited by 90
  10. 2025Memp: Exploring Agent Procedural Memorycited by 88
  11. 2024DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositoriescited by 71
  12. 2024SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositoriescited by 52
5 more
  1. 2025Repo2Run: Automated Building Executable Environment for Code Repository at Scalecited by 39
  2. 2025EnvBench: A Benchmark for Automated Environment Setupcited by 36
  3. 2026SkillCraft: Can LLM Agents Learn to Use Tools Skillfully?cited by 28
  4. 2026Agent-as-a-Judgecited by 15
  5. 2019An Empirical Analysis of the Python Package Index (PyPI)cited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026RepoRescue: An Empirical Study of LLM Agents on Whole-Repository Compatibility Rescue
  2. 2025One Tool Is Enough: Reinforcement Learning for Repository-Level LLM Agents
  3. 2025SetupBench: Assessing Software Engineering Agents' Ability to Bootstrap Development Environments
  4. 2026Probe-and-Refine Tuning of Repository Guidance for Coding Agents
  5. 2026Needle in the Repo: A Benchmark for Maintainability in AI-Generated Repository Edits
  6. 2025HAFixAgent: History-Aware Program Repair Agent
  7. 2026FC-SWE: Failure-Conditioned RL for Long-Horizon Software Engineering Agents
  8. 2026Large-scale Repository Engineering via Agent-Native Reusable Code Primitives