# Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLMs

arXiv: https://arxiv.org/abs/2506.19290 · HTML: https://mlsanity.com/paper/2506.19290/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 26 of its 45 references that are in the map.
- First posted on arXiv 24 Jun 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 26 of 26)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [WizardCoder: Empowering Code Large Language Models with Evol-Instruct](https://mlsanity.com/paper/2306.08568/) (2023)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions](https://mlsanity.com/paper/2406.15877/) (2024)
- [Agentless: Demystifying LLM-based Software Engineering Agents](https://mlsanity.com/paper/2407.01489/) (2024)
- [Training Software Engineering Agents and Verifiers with SWE-Gym](https://mlsanity.com/paper/2412.21139/) (2024)
- [SWE-smith: Scaling Data for Software Engineering Agents](https://mlsanity.com/paper/2504.21798/) (2025)
- [AutoCodeRover: Autonomous Program Improvement](https://mlsanity.com/paper/2404.05427/) (2024)
- [Sequence Parallelism: Long Sequence Training from System Perspective](https://mlsanity.com/paper/2105.13120/) (2021)
- [SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution](https://mlsanity.com/paper/2501.05040/) (2025)
- [Thinking Longer, Not Larger: Enhancing Software Engineering Agents via Scaling Test-Time Compute](https://mlsanity.com/paper/2503.23803/) (2025)
- [SWE-Dev: Building Software Engineering Agents with Training and Inference Scaling](https://mlsanity.com/paper/2506.07636/) (2025)

## Built on by (most cited first, 26 of 26)

- [Kimi-Dev: Agentless Training as Skill Prior for SWE-Agents](https://mlsanity.com/paper/2509.23045/) (2025)
- [SWE-Lego: Pushing the Limits of Supervised Fine-tuning for Software Issue Resolving](https://mlsanity.com/paper/2601.01426/) (2026)
- [BugPilot: Complex Bug Generation for Efficient Learning of SWE Skills](https://mlsanity.com/paper/2510.19898/) (2025)
- [SWE-World: Building Software Engineering Agents in Docker-Free Environments](https://mlsanity.com/paper/2602.03419/) (2026)
- [SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training](https://mlsanity.com/paper/2602.03411/) (2026)
- [Immersion in the GitHub Universe: Scaling Coding Agents to Mastery](https://mlsanity.com/paper/2602.09892/) (2026)
- [Imitation Learning for Multi-turn LM Agents via On-policy Expert Corrections](https://mlsanity.com/paper/2512.14895/) (2025)
- [daVinci-Env: Open SWE Environment Synthesis at Scale](https://mlsanity.com/paper/2603.13023/) (2026)
- [SERA: Soft-Verified Efficient Repository Agents](https://mlsanity.com/paper/2601.20789/) (2026)
- [Open-SWE-Traces: Advancing Dual-Mode Multilingual Distillation for Software Engineering Agents](https://mlsanity.com/paper/2606.16038/) (2026)
- [Klear-AgentForge: Forging Agentic Intelligence through Posttraining Scaling](https://mlsanity.com/paper/2511.05951/) (2025)
- [RepoForge: Training a SOTA Fast-thinking SWE Agent with an End-to-End Data Curation Pipeline Synergizing SFT and RL at Scale](https://mlsanity.com/paper/2508.01550/) (2025)
- [Orchard: An Open-Source Agentic Modeling Framework](https://mlsanity.com/paper/2605.15040/) (2026)
- [G-LNS: Generative Large Neighborhood Search for LLM-Based Automatic Heuristic Design](https://mlsanity.com/paper/2602.08253/) (2026)
- [Building Coding Agents via Entropy-Enhanced Multi-Turn Preference Optimization](https://mlsanity.com/paper/2509.12434/) (2025)
- [Agentic Software Issue Resolution with Large Language Models: A Survey](https://mlsanity.com/paper/2512.22256/) (2025)
- [SWE-AGILE: A Software Agent Framework for Efficiently Managing Dynamic Reasoning Context](https://mlsanity.com/paper/2604.11716/) (2026)
- [SWE-Hub: A Unified Production System for Scalable, Executable Software Engineering Tasks](https://mlsanity.com/paper/2603.00575/) (2026)
- [From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents](https://mlsanity.com/paper/2605.21996/) (2026)
- [What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents](https://mlsanity.com/paper/2608.27260/) (2026)
- [Don't Claim Benchmark-Oriented Optimization Improves General Coding Capability -- Diverse Evaluation Is Required](https://mlsanity.com/paper/2608.13566/) (2026)
- [SWE-Fuse: Empowering Software Agents via Issue-free Trajectory Learning and Entropy-aware RLVR Training](https://mlsanity.com/paper/2603.07927/) (2026)
- [PIPer: On-Device Environment Setup via Online Reinforcement Learning](https://mlsanity.com/paper/2509.25455/) (2025)
- [One to More, More to One: Category-Aware Iterative Expert Training for Software Engineering Agents](https://mlsanity.com/paper/2609.23377/) (2026)
- [ANTE: Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefs](https://mlsanity.com/paper/2610.15368/) (2026)
- [Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards](https://mlsanity.com/paper/2610.18868/) (2026)

## Abstract (arXiv)

Software engineering (SWE) has recently emerged as a crucial testbed for next-generation LLM agents, demanding inherent capabilities in two critical dimensions: sustained iterative problem-solving (e.g., >50 interaction rounds) and long-context dependency resolution (e.g., >32k tokens). However, the data curation process in SWE remains notoriously time-consuming, as it heavily relies on manual annotation for code file filtering and the setup of dedicated runtime environments to execute and validate unit tests. Consequently, most existing datasets are limited to only a few thousand GitHub-sourced instances. To this end, we propose an incremental, automated data-curation pipeline that systematically scales both the volume and diversity of SWE datasets. Our dataset comprises 10,169 real-world Python task instances from 2,531 distinct GitHub repositories, each accompanied by a task specified in natural language and a dedicated runtime-environment image for automated unit-test validation. We have carefully curated over 8,000 successfully runtime-validated training trajectories from our proposed SWE dataset. When fine-tuning the Skywork-SWE model on these trajectories, we uncover a striking data scaling phenomenon: the trained model's performance for software engineering capabilities in LLMs continues to improve as the data size increases, showing no signs of saturation. Notably, our Skywork-SWE model achieves 38.0% pass@1 accuracy on the SWE-bench Verified benchmark without using verifiers or multiple rollouts, establishing a new state-of-the-art (SOTA) among the Qwen2.5-Coder-32B-based LLMs built on the OpenHands agent framework. Furthermore, with the incorporation of test-time scaling techniques, the performance further improves to 47.0% accuracy, surpassing the previous SOTA results for sub-32B parameter models. We release the Skywork-SWE-32B model checkpoint to accelerate future research.