SWE-Universe: Scale Real-World Verifiable Environments to Millions
Not on the map. Why?
Abstract (arXiv)
We propose SWE-Universe, a scalable and efficient framework for automatically constructing real-world software engineering (SWE) verifiable environments from GitHub pull requests (PRs). To overcome the prevalent challenges of automatic building, such as low production yield, weak verifiers, and prohibitive cost, our framework utilizes a building agent powered by an efficient custom-trained model. This agent employs iterative self-verification and in-loop hacking detection to ensure the reliable generation of high-fidelity, verifiable tasks. Using this method, we scale the number of real-world multilingual SWE environments to a million scale (807,693). We demonstrate the profound value of our environments through large-scale agentic mid-training and reinforcement learning. Finally, we applied this technique to Qwen3-Max-Thinking and achieved a score of 75.3% on SWE-Bench Verified. Our work provides both a critical resource and a robust methodology to advance the next generation of coding agents.
Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 40 of its 52 references that are in the map. First posted on arXiv 2 Feb 2026.
Builds on
40 of 52 references.
References · 40
- 2025Qwen3 Technical Reportcited by 8,195
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
- 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
- 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 300
- 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 255
- 2023An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generationcited by 193
- 2026MiMo-V2-Flash Technical Reportcited by 163
- 2025SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agentscited by 127
- 2023InferFix: End-to-End Program Repair with LLMscited by 113
- 2018NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating Systemcited by 97
28 more
- 2024SWE-Bench+: Enhanced Coding Benchmark for LLMscited by 95
- 2023ChatUniTest: A Framework for LLM-Based Test Generationcited by 94
- 2025CWM: An Open-Weights LLM for Research on Code Generation with World Modelscited by 87
- 2025SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agentscited by 80
- 2025Agent Learning via Early Experiencecited by 79
- 2025SWE-bench Goes Live!cited by 74
- 2022Pynguin: Automated Unit Test Generation for Pythoncited by 49
- 2024You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projectscited by 45
- 2026Endless Terminals: Scaling RL Environments for Terminal Agentscited by 43
- 2025Repo2Run: Automated Building Executable Environment for Code Repository at Scalecited by 39
- 2025EnvBench: A Benchmark for Automated Environment Setupcited by 36
- 2023A3Test: Assertion-Augmented Automated Test Case Generationcited by 33
- 2024Fewer Truncations Improve Language Modelingcited by 33
- 2025Dynamic Scaling of Unit Tests for Code Reward Modelingcited by 31
- 2025BugPilot: Complex Bug Generation for Efficient Learning of SWE Skillscited by 28
- 2025Automated Benchmark Generation for Repository-Level Coding Taskscited by 27
- 2025SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositoriescited by 27
- 2024CodeBenchGen: Creating Scalable Execution-based Code Generation Benchmarkscited by 23
- 2024Beyond pip install: Evaluating LLM Agents for the Automated Installation of Python Projectscited by 21
- 2025SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Mannercited by 18
- 2025SetupBench: Assessing Software Engineering Agents' Ability to Bootstrap Development Environmentscited by 18
- 2025Process-Level Trajectory Evaluation for Environment Configuration in Software Engineering Agentscited by 17
- 2026daVinci-Dev: Agent-native Mid-training for Software Engineeringcited by 15
- 2019DockerizeMe: Automatic Inference of Environment Dependencies for Python Code Snippetscited by 13
- 2024B4: Towards Optimal Assessment of Plausible Code Solutions with Plausible Testscited by 9
- 2026MegaFlow: Large-Scale Distributed Orchestration System for the Agentic Eracited by 9
- 2025Multi-Docker-Eval: A `Shovel of the Gold Rush' Benchmark on Automatic Environment Building for Software Engineeringcited by 7
- 2026EvoConfig: Self-Evolving Multi-Agent Systems for Efficient Autonomous Environment Configurationcited by 7
Built on by
Citing papers per year: 2026: 16.
16 papers.
Sort:
By area:
- 2026Qwen3-Coder-Next Technical Reportcited by 60
- 2026Tmax: A simple recipe for terminal agentscited by 24
- 2026Qwen-AgentWorld: Language World Models for General Agentscited by 22
- 2026CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agentscited by 21
- 2026daVinci-Env: Open SWE Environment Synthesis at Scalecited by 13
- 2026SWE-MeM: Learning Adaptive Memory Management for Long-Horizon Coding Agentscited by 12
- 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
- 2026The Verification Horizon: No Silver Bullet for Coding Agent Rewardscited by 6
- 2026LEGO-RL: Harness-Native Reinforcement Learning for Coding Agentscited by 3
- 2026ScienceIDE: Turning World's Scientific Codebase into Agent Learnable Environmentscited by 2
- 2026Meta-Task: Turning Terminal Task Synthesis into a Terminal Task for Scalable Agent Trainingcited by 1
- 2026PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarkscited by 1
4 more
- 2026Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environmentscited by 1
- 2026CodeMidas: Scaling Agentic Coding RL Environments from Code Itselfcited by 1
- 2026Reasoning4Sciences: Bridging Reasoning Language Models to All Scientific Branchescited by 0
- 2026SWE-MILE: Asynchronous Potential-Induced Milestone Credit Assignment for Long-Horizon Software Engineering Agentscited by 0
- 2026SWE-MILE: Asynchronous Potential-Induced Milestone Credit Assignment for Long-Horizon Software Engineering Agents26 Sep 2026
- 2026CodeMidas: Scaling Agentic Coding RL Environments from Code Itself18 Sep 2026
- 2026ScienceIDE: Turning World's Scientific Codebase into Agent Learnable Environments16 Sep 2026
- 2026LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents18 Aug 2026
- 2026Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments30 Jul 2026
- 2026PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks30 Jul 2026
- 2026Meta-Task: Turning Terminal Task Synthesis into a Terminal Task for Scalable Agent Training30 Jul 2026
- 2026SWE-MeM: Learning Adaptive Memory Management for Long-Horizon Coding Agents26 Jun 2026
- 2026The Verification Horizon: No Silver Bullet for Coding Agent Rewards24 Jun 2026
- 2026Qwen-AgentWorld: Language World Models for General Agents23 Jun 2026
- 2026Tmax: A simple recipe for terminal agents22 Jun 2026
- 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application10 Jun 2026
4 more
- 2026Reasoning4Sciences: Bridging Reasoning Language Models to All Scientific Branches31 May 2026
- 2026CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents25 May 2026
- 2026daVinci-Env: Open SWE Environment Synthesis at Scale13 Mar 2026
- 2026Qwen3-Coder-Next Technical Report28 Feb 2026
- 2026Qwen3-Coder-Next Technical Reportcited by 60
- 2026Tmax: A simple recipe for terminal agentscited by 24
- 2026CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agentscited by 21
- 2026SWE-MeM: Learning Adaptive Memory Management for Long-Horizon Coding Agentscited by 12
- 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
- 2026The Verification Horizon: No Silver Bullet for Coding Agent Rewardscited by 6
- 2026LEGO-RL: Harness-Native Reinforcement Learning for Coding Agentscited by 3
- 2026Meta-Task: Turning Terminal Task Synthesis into a Terminal Task for Scalable Agent Trainingcited by 1
- 2026Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environmentscited by 1
- 2026CodeMidas: Scaling Agentic Coding RL Environments from Code Itselfcited by 1
- 2026SWE-MILE: Asynchronous Potential-Induced Milestone Credit Assignment for Long-Horizon Software Engineering Agentscited by 0
- 2026SWE-MILE: Asynchronous Potential-Induced Milestone Credit Assignment for Long-Horizon Software Engineering Agents26 Sep 2026
- 2026CodeMidas: Scaling Agentic Coding RL Environments from Code Itself18 Sep 2026
- 2026LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents18 Aug 2026
- 2026Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments30 Jul 2026
- 2026Meta-Task: Turning Terminal Task Synthesis into a Terminal Task for Scalable Agent Training30 Jul 2026
- 2026SWE-MeM: Learning Adaptive Memory Management for Long-Horizon Coding Agents26 Jun 2026
- 2026The Verification Horizon: No Silver Bullet for Coding Agent Rewards24 Jun 2026
- 2026Tmax: A simple recipe for terminal agents22 Jun 2026
- 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application10 Jun 2026
- 2026CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents25 May 2026
- 2026Qwen3-Coder-Next Technical Report28 Feb 2026
- 2026Qwen-AgentWorld: Language World Models for General Agentscited by 22
- 2026Qwen-AgentWorld: Language World Models for General Agents23 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026SWE-World: Building Software Engineering Agents in Docker-Free Environments
- 2026Immersion in the GitHub Universe: Scaling Coding Agents to Mastery
- 2026SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale
- 2025R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents
- 2025SWE-Dev: Building Software Engineering Agents with Training and Inference Scaling
- 2026E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch
- 2025Training Versatile Coding Agents in Synthetic Environments
- 2026SWE-Next: Scalable Real-World Software Engineering Tasks for Agents