# RepoRescue: An Empirical Study of LLM Agents on Whole-Repository Compatibility Rescue arXiv: https://arxiv.org/abs/2607.01213 ยท HTML: https://mlsanity.com/paper/2607.01213/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. - Builds on 37 of its 54 references that are in the map. - First posted on arXiv 1 Jul 2026. ## Placed in - LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/) ## Builds on (most cited first, 30 of 37) - [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021) - [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023) - [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024) - [Agentless: Demystifying LLM-based Software Engineering Agents](https://mlsanity.com/paper/2407.01489/) (2024) - [CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges](https://mlsanity.com/paper/2401.07339/) (2024) - [RepairAgent: An Autonomous, LLM-Based Agent for Program Repair](https://mlsanity.com/paper/2403.17134/) (2024) - [On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub](https://mlsanity.com/paper/2509.14745/) (2025) - [RepoAudit: An Autonomous LLM-Agent for Repository-Level Code Auditing](https://mlsanity.com/paper/2501.18160/) (2025) - [You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projects](https://mlsanity.com/paper/2412.10133/) (2024) - [UniDebugger: Hierarchical Multi-Agent Framework for Unified Software Debugging](https://mlsanity.com/paper/2404.17153/) (2024) - [Stop Comparing LLM Agents Without Disclosing the Harness](https://mlsanity.com/paper/2605.23950/) (2026) - [Enhancing repository-level software repair via repository-aware knowledge graphs](https://mlsanity.com/paper/2503.21710/) (2025) - [Do Developers Update Their Library Dependencies? An Empirical Study on the Impact of Security Advisories on Library Migration](https://mlsanity.com/paper/1709.04621/) (2017) - [An Empirical Comparison of Dependency Network Evolution in Seven Software Packaging Ecosystems](https://mlsanity.com/paper/1710.04936/) (2017) - [PyCG: Practical Call Graph Generation in Python](https://mlsanity.com/paper/2103.00587/) (2021) - [CODEMENV: Benchmarking Large Language Models on Code Migration](https://mlsanity.com/paper/2506.00894/) (2025) - [When Large Language Models Confront Repository-Level Automatic Program Repair: How Well They Done?](https://mlsanity.com/paper/2403.00448/) (2024) - [Scalpel: The Python Static Analysis Framework](https://mlsanity.com/paper/2202.11840/) (2022) - [SGAgent: Suggestion-Guided LLM-Based Multi-Agent Framework for Repository-Level Software Repair](https://mlsanity.com/paper/2602.23647/) (2026) - [Why Modern Open Source Projects Fail](https://mlsanity.com/paper/1707.02327/) (2017) - [Demystifying the Vulnerability Propagation and Its Evolution via Dependency Trees in the NPM Ecosystem](https://mlsanity.com/paper/2201.03981/) (2022) - [RepoRepair: Leveraging Code Documentation for Repository-Level Automated Program Repair](https://mlsanity.com/paper/2603.01048/) (2026) - [On the evolution of technical lag in the npm package dependency network](https://mlsanity.com/paper/1806.01545/) (2018) - [On the abandonment and survival of open source projects: An empirical investigation](https://mlsanity.com/paper/1906.08058/) (2019) - [PSearch: Search-based Patch Generation in the Era of LLM-based Automated Program Repair](https://mlsanity.com/paper/2507.01827/) (2025) - [Uncovering Systematic Failures of LLMs in Verifying Code Against Natural Language Specifications](https://mlsanity.com/paper/2508.12358/) (2025) - [FreshBrew: A Benchmark for Evaluating AI Agents on Java Code Migration](https://mlsanity.com/paper/2510.04852/) (2025) - [AI-Generated Code Is Not Reproducible (Yet): An Empirical Study of Dependency Gaps in LLM-Based Coding Agents](https://mlsanity.com/paper/2512.22387/) (2025) - [RGFL: Reasoning Guided Fault Localization for Automated Program Repair Using Large Language Models](https://mlsanity.com/paper/2601.18044/) (2026) - [The Last Dependency Crusade: Solving Python Dependency Conflicts with LLMs](https://mlsanity.com/paper/2501.16191/) (2025) ## Built on by (most cited first, 1 of 1) - [Repo0: Design-Driven Zero-to-All Code Generation](https://mlsanity.com/paper/2608.19854/) (2026) ## Abstract (arXiv) Open-source libraries and tools are widely reused, but compatibility maintenance is expensive. Once maintainers leave, useful repositories can stop working as runtimes and dependencies evolve. We study whether LLM agents can adapt old repositories to modern environments, a task we call compatibility rescue. Unlike bug repair, compatibility rescue starts from a repository that worked in its original environment but fails after ecosystem drift. RepoRescue gives agents only the repository and its failing modern environment; the agent must diagnose the failure, locate affected code, and produce a source-code rescue that restores the historical test suite. We build RepoRescue from 193 Python and 122 Java repositories, each verified to pass historically and fail after modernization. We evaluate five deployed agent systems on Python and three on Java. Beyond full-patch pass rate, we rerun patches after removing test-file edits to measure source-only repair, add a runtime-enforced regime that blocks test edits, and validate practical use for repositories whose suites pass after rescue. We find that Claude Code systems sometimes edit failing tests even when prompted not to; with runtime blocking, Kimi still rescues 41.5% of repositories. Systems are complementary: their union reaches 62.7%, exceeding the best single system by 10.9 points. Difficulty concentrates in cross-file coordination: on 14 repositories requiring coordinated whole-codebase changes, GPT-5.2 through Codex passes all 14, while every Claude Code system passes at most two. Finally, a passing suite is only an initial signal: among 34 unmaintained Python candidates whose suites pass after rescue, 22 work in realistic scenarios and 12 pass bug-hunt with patches that address the compatibility failure. RepoRescue benchmarks compatibility rescue with source-only auditing, runtime enforcement, practical validation, and reasoning labels.