# SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale

arXiv: https://arxiv.org/abs/2602.23866 · HTML: https://mlsanity.com/paper/2602.23866/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 45 references that are in the map.
- First posted on arXiv 27 Feb 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 17 of 17)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [Training Software Engineering Agents and Verifiers with SWE-Gym](https://mlsanity.com/paper/2412.21139/) (2024)
- [SWE-smith: Scaling Data for Software Engineering Agents](https://mlsanity.com/paper/2504.21798/) (2025)
- [Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving](https://mlsanity.com/paper/2504.02605/) (2025)
- [Scaling Long-Horizon LLM Agent via Context-Folding](https://mlsanity.com/paper/2510.11967/) (2025)
- [SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents](https://mlsanity.com/paper/2505.20411/) (2025)
- [SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agents](https://mlsanity.com/paper/2504.08703/) (2025)
- [Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models](https://mlsanity.com/paper/2512.13607/) (2025)
- [Toward Training Superintelligent Software Agents through Self-Play SWE-RL](https://mlsanity.com/paper/2512.18552/) (2025)
- [SWE-bench-java: A GitHub Issue Resolving Benchmark for Java](https://mlsanity.com/paper/2408.14354/) (2024)
- [Automated Benchmark Generation for Repository-Level Coding Tasks](https://mlsanity.com/paper/2503.07701/) (2025)
- [SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositories](https://mlsanity.com/paper/2512.17419/) (2025)
- [SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Manner](https://mlsanity.com/paper/2506.09003/) (2025)
- [SPICE: An Automated SWE-Bench Labeling Pipeline for Issue Clarity, Test Coverage, and Effort Estimation](https://mlsanity.com/paper/2507.09108/) (2025)
- [RepoForge: Training a SOTA Fast-thinking SWE Agent with an End-to-End Data Curation Pipeline Synergizing SFT and RL at Scale](https://mlsanity.com/paper/2508.01550/) (2025)
- [SWE-Sharp-Bench: A Reproducible Benchmark for C# Software Engineering Tasks](https://mlsanity.com/paper/2511.02352/) (2025)

## Built on by (most cited first, 29 of 29)

- [Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning](https://mlsanity.com/paper/2606.15007/) (2026)
- [SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks](https://mlsanity.com/paper/2603.24755/) (2026)
- [Open-SWE-Traces: Advancing Dual-Mode Multilingual Distillation for Software Engineering Agents](https://mlsanity.com/paper/2606.16038/) (2026)
- [Dockerless: Environment-Free Program Verifier for Coding Agents](https://mlsanity.com/paper/2606.28436/) (2026)
- [KAT-Coder-V2 Technical Report](https://mlsanity.com/paper/2603.27703/) (2026)
- [Orchard: An Open-Source Agentic Modeling Framework](https://mlsanity.com/paper/2605.15040/) (2026)
- [A Framework for Evaluating Agentic Skills at Scale](https://mlsanity.com/paper/2606.17819/) (2026)
- [OpenForgeRL: Train Harness-native Agents in Any Environment](https://mlsanity.com/paper/2607.21557/) (2026)
- [ChainSWE: Benchmarking Coding Agents on Multi-Bug Software Maintenance](https://mlsanity.com/paper/2607.02606/) (2026)
- [Intern-S2-Preview: Scientific Agentic Foundation Model](https://mlsanity.com/paper/2608.13505/) (2026)
- [Position: Coding Benchmarks Are Misaligned with Agentic Software Engineering](https://mlsanity.com/paper/2606.17799/) (2026)
- [What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents](https://mlsanity.com/paper/2608.27260/) (2026)
- [RepoLaunch: Automating Build and Management of Code Repositories across Languages and Platforms](https://mlsanity.com/paper/2603.05026/) (2026)
- [Yet Even Less Is Even Better For Agentic, Reasoning, and Coding LLMs](https://mlsanity.com/paper/2604.00824/) (2026)
- [Agentic Agile-V: From Vibe Coding to Verified Engineering in Software and Hardware Development](https://mlsanity.com/paper/2605.20456/) (2026)
- [CodeMidas: Scaling Agentic Coding RL Environments from Code Itself](https://mlsanity.com/paper/2609.22068/) (2026)
- [Terminal Agents: A Survey of AI Agents in Command-Line Environments](https://mlsanity.com/paper/2608.20485/) (2026)
- [MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement](https://mlsanity.com/paper/2610.11959/) (2026)
- [Binocular Gaze Estimation with Single Camera and Single Light Source](https://mlsanity.com/paper/2607.05473/) (2026)
- [One to More, More to One: Category-Aware Iterative Expert Training for Software Engineering Agents](https://mlsanity.com/paper/2609.23377/) (2026)
- [VibeMemBench: Evaluating Memory Systems for Coding Agents on Real Repository Coding Tasks](https://mlsanity.com/paper/2609.23570/) (2026)
- [SWE-MILE: Asynchronous Potential-Induced Milestone Credit Assignment for Long-Horizon Software Engineering Agents](https://mlsanity.com/paper/2609.32631/) (2026)
- [AsynCodeBench: Benchmarking Collaboration of Asynchronous Multi-Agent Systems in Software Engineering](https://mlsanity.com/paper/2609.32662/) (2026)
- [Counterfactual Rollout Replay: Forkable Environments as Free Process Rewards for Software Engineering Agents](https://mlsanity.com/paper/2609.33875/) (2026)
- [Re-executing Coding-Agent Evaluations: Unchanged Decisions and Unresolved Policy Risk](https://mlsanity.com/paper/2610.14460/) (2026)
- [CodeWalk: Generating Coding Benchmarks by Walking a Problem Graph](https://mlsanity.com/paper/2610.15345/) (2026)
- [ANCHOR: Assessing Naturalistic Constraint Hierarchies and Ordering in Repository-Grounded Agentic Coding](https://mlsanity.com/paper/2610.16859/) (2026)
- [SWE-Nexus: Automated Composition of Software Engineering Tasks from Test Execution](https://mlsanity.com/paper/2610.20038/) (2026)
- [SWE-Stream: Scaling Executable Training Data for SWE Agents via Progressive Construction and Elastic Streaming](https://mlsanity.com/paper/2610.20193/) (2026)

## Abstract (arXiv)

Software engineering agents (SWE) are improving rapidly, with recent gains largely driven by reinforcement learning (RL). However, RL training is constrained by the scarcity of large-scale task collections with reproducible execution environments and reliable test suites. Although a growing number of benchmarks have emerged, datasets suitable for training remain limited in scale and diversity or often target a limited set of high-resource language ecosystems. We introduce SWE-rebench V2, a language-agnostic automated pipeline for harvesting executable real-world SWE tasks and constructing RL training environments at scale. The pipeline synthesizes repository-specific installation and test procedures via an interactive setup agent, and filters unsound instances using an ensemble of LLM judges, validated against human-verified SWE-bench annotations. Using this pipeline, we construct a dataset of 32,079 tasks spanning 20 languages and 3,617 repositories, with pre-built images for reproducible execution. To further scale training data, we additionally release 120,000+ tasks with installation instructions, fail-to-pass tests and rich metadata, where the problem statement is generated based on the original pull request description. We validate the collected instances through a diagnostic study that covers a subset of tasks in five programming languages across seven popular models, and provide instance-level metadata that flags common confounders such as overly restrictive tests and underspecified descriptions. We release the datasets, the collection and execution code, and associated artifacts to enable large-scale training of SWE agents across diverse languages and repositories.