# BeyondSWE: Can Current Code Agent Survive Beyond Single-Repo Bug Fixing?

arXiv: https://arxiv.org/abs/2603.03194 · HTML: https://mlsanity.com/paper/2603.03194/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 40 of its 70 references that are in the map.
- First posted on arXiv 3 Mar 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 30 of 40)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://mlsanity.com/paper/2512.02556/) (2025)
- [GLM-5: from Vibe Coding to Agentic Engineering](https://mlsanity.com/paper/2602.15763/) (2026)
- [Kimi K2: Open Agentic Intelligence](https://mlsanity.com/paper/2507.20534/) (2025)
- [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026)
- [Agentless: Demystifying LLM-based Software Engineering Agents](https://mlsanity.com/paper/2407.01489/) (2024)
- [Training Software Engineering Agents and Verifiers with SWE-Gym](https://mlsanity.com/paper/2412.21139/) (2024)
- [SWE-smith: Scaling Data for Software Engineering Agents](https://mlsanity.com/paper/2504.21798/) (2025)
- [SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?](https://mlsanity.com/paper/2509.16941/) (2025)
- [AutoCodeRover: Autonomous Program Improvement](https://mlsanity.com/paper/2404.05427/) (2024)
- [SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution](https://mlsanity.com/paper/2502.18449/) (2025)
- [SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?](https://mlsanity.com/paper/2410.03859/) (2024)
- [MiMo-V2-Flash Technical Report](https://mlsanity.com/paper/2601.02780/) (2026)
- [SciCode: A Research Coding Benchmark Curated by Scientists](https://mlsanity.com/paper/2407.13168/) (2024)
- [Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving](https://mlsanity.com/paper/2504.02605/) (2025)
- [R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents](https://mlsanity.com/paper/2504.07164/) (2025)
- [SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents](https://mlsanity.com/paper/2505.20411/) (2025)
- [SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement](https://mlsanity.com/paper/2410.20285/) (2024)
- [SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agents](https://mlsanity.com/paper/2504.08703/) (2025)
- [SWE-bench Goes Live!](https://mlsanity.com/paper/2505.23419/) (2025)
- [WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agents](https://mlsanity.com/paper/2509.13309/) (2025)
- [Tongyi DeepResearch Technical Report](https://mlsanity.com/paper/2510.24701/) (2025)
- [NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents](https://mlsanity.com/paper/2512.12730/) (2025)
- [Seed-Coder: Let the Code Model Curate Data for Itself](https://mlsanity.com/paper/2506.03524/) (2025)
- [The OpenHands Software Agent SDK: A Composable and Extensible Foundation for Production Agents](https://mlsanity.com/paper/2511.03690/) (2025)
- [Kimi-Dev: Agentless Training as Skill Prior for SWE-Agents](https://mlsanity.com/paper/2509.23045/) (2025)
- [SWE-Mirror: Scaling Issue-Resolving Datasets by Mirroring Issues Across Repositories](https://mlsanity.com/paper/2509.08724/) (2025)
- [SWE-Lego: Pushing the Limits of Supervised Fine-tuning for Software Issue Resolving](https://mlsanity.com/paper/2601.01426/) (2026)

## Built on by (most cited first, 22 of 22)

- [SWE-Explore: Benchmarking How Coding Agents Explore Repositories](https://mlsanity.com/paper/2606.07297/) (2026)
- [Toward Autonomous Long-Horizon Engineering for ML Research](https://mlsanity.com/paper/2604.13018/) (2026)
- [daVinci-Env: Open SWE Environment Synthesis at Scale](https://mlsanity.com/paper/2603.13023/) (2026)
- [AI for Auto-Research: Roadmap & User Guide](https://mlsanity.com/paper/2605.18661/) (2026)
- [Open-SWE-Traces: Advancing Dual-Mode Multilingual Distillation for Software Engineering Agents](https://mlsanity.com/paper/2606.16038/) (2026)
- [Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills](https://mlsanity.com/paper/2606.07412/) (2026)
- [CalibForge: Adversarial Solver Calibration for Scaling Learnable Terminal Tasks](https://mlsanity.com/paper/2608.06352/) (2026)
- [REAgent: Requirement-Driven LLM Agents for Software Issue Resolution](https://mlsanity.com/paper/2604.06861/) (2026)
- [Evolve as a Team: Collaborative Self-Evolution for LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2605.29790/) (2026)
- [ChainSWE: Benchmarking Coding Agents on Multi-Bug Software Maintenance](https://mlsanity.com/paper/2607.02606/) (2026)
- [LoopsBench: From Harness Engineering to Loop Engineering in Coding Agent Evaluation](https://mlsanity.com/paper/2608.00267/) (2026)
- [LoopArena: Benchmarking Models as Runtime Controllers for Loop Engineering](https://mlsanity.com/paper/2608.28281/) (2026)
- [OctoLong: Mid-Training On Cross-Repository Code Contexts Enhances Long-Context Modeling](https://mlsanity.com/paper/2608.05141/) (2026)
- [SkillForge: Self-Distilling Agents for Project-Specific Issue Resolution](https://mlsanity.com/paper/2608.18933/) (2026)
- [Repo-To-Skill: Distilling GitHub Repositories Into AI4AI Skills](https://mlsanity.com/paper/2609.02749/) (2026)
- [Shortcutting the Fix: Identifying and Categorizing Agentic Exploits in Software Engineering Benchmarks](https://mlsanity.com/paper/2609.06780/) (2026)
- [Alipay-PIBench: A Realistic Payment Integration Benchmark for Coding Agents](https://mlsanity.com/paper/2607.14573/) (2026)
- [Update from Hell: Can Coding Agents Survive Hidden Breakage in Dependency Upgrades?](https://mlsanity.com/paper/2608.30300/) (2026)
- [Agent-Editing World Model: Rethinking World Modeling for LLM Agents](https://mlsanity.com/paper/2609.28416/) (2026)
- [SWE-MILE: Asynchronous Potential-Induced Milestone Credit Assignment for Long-Horizon Software Engineering Agents](https://mlsanity.com/paper/2609.32631/) (2026)
- [E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch](https://mlsanity.com/paper/2609.38335/) (2026)
- [AECP: Artifact-Exclusive Communication Protocol for Multi-Agent Code Generation](https://mlsanity.com/paper/2610.06481/) (2026)

## Abstract (arXiv)

Current code-agent benchmarks primarily evaluate localized issue resolution within a single target repository, leaving under-tested many software engineering tasks that require external knowledge or broader repository-level changes. We introduce BeyondSWE, a 500-instance benchmark drawn from 246 real-world GitHub repositories to evaluate code agents beyond single-repository bug fixing. BeyondSWE covers four representative settings: cross-repository issue resolution, domain-specific issue resolution, dependency-driven migration, and document-to-repository generation, spanning both broader knowledge scope and broader resolution scope. Our evaluation shows that BeyondSWE remains far from saturated: the best OpenHands-based agent reaches 46.12 average score, while the strongest Codex harness with GPT-5.4 (xhigh) reaches 56.65 under a search-aware prompt. To study whether external information access closes this gap, we use SearchSWE as a controlled diagnostic baseline for search-augmented coding. Search access improves most models and substantially helps some tasks, but the gains remain limited and uneven, showing that current agents still struggle to convert retrieved information into precise, version-compatible, and locally actionable code changes. These results suggest that deep search for coding remains an open problem: progress requires agents that can reliably combine external evidence with repository-local reasoning and execution-based verification.