# SWE-Lego: Pushing the Limits of Supervised Fine-tuning for Software Issue Resolving

arXiv: https://arxiv.org/abs/2601.01426 · HTML: https://mlsanity.com/paper/2601.01426/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 52 references that are in the map.
- First posted on arXiv 4 Jan 2026.

## Builds on (most cited first, 30 of 34)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [WizardCoder: Empowering Code Large Language Models with Evol-Instruct](https://mlsanity.com/paper/2306.08568/) (2023)
- [Generative Verifiers: Reward Modeling as Next-Token Prediction](https://mlsanity.com/paper/2408.15240/) (2024)
- [The Stack: 3 TB of permissively licensed source code](https://mlsanity.com/paper/2211.15533/) (2022)
- [RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems](https://mlsanity.com/paper/2306.03091/) (2023)
- [Training Software Engineering Agents and Verifiers with SWE-Gym](https://mlsanity.com/paper/2412.21139/) (2024)
- [AutoCodeRover: Autonomous Program Improvement](https://mlsanity.com/paper/2404.05427/) (2024)
- [Magicoder: Empowering Code Generation with OSS-Instruct](https://mlsanity.com/paper/2312.02120/) (2023)
- [InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback](https://mlsanity.com/paper/2306.14898/) (2023)
- [SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?](https://mlsanity.com/paper/2410.03859/) (2024)
- [OctoPack: Instruction Tuning Code Large Language Models](https://mlsanity.com/paper/2308.07124/) (2023)
- [CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion](https://mlsanity.com/paper/2310.11248/) (2023)
- [ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation](https://mlsanity.com/paper/2308.01861/) (2023)
- [R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents](https://mlsanity.com/paper/2504.07164/) (2025)
- [SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents](https://mlsanity.com/paper/2505.20411/) (2025)
- [CodePlan: Repository-level Coding using LLMs and Planning](https://mlsanity.com/paper/2309.12499/) (2023)
- [Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?](https://mlsanity.com/paper/2511.13646/) (2025)
- [CWM: An Open-Weights LLM for Research on Code Generation with World Models](https://mlsanity.com/paper/2510.02387/) (2025)
- [CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models](https://mlsanity.com/paper/2404.13161/) (2024)
- [Kimi-Dev: Agentless Training as Skill Prior for SWE-Agents](https://mlsanity.com/paper/2509.23045/) (2025)
- [RepoFusion: Training Code Models to Understand Your Repository](https://mlsanity.com/paper/2306.10998/) (2023)
- [Toward Training Superintelligent Software Agents through Self-Play SWE-RL](https://mlsanity.com/paper/2512.18552/) (2025)

## Built on by (most cited first, 30 of 31)

- [SWE-World: Building Software Engineering Agents in Docker-Free Environments](https://mlsanity.com/paper/2602.03419/) (2026)
- [SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training](https://mlsanity.com/paper/2602.03411/) (2026)
- [Immersion in the GitHub Universe: Scaling Coding Agents to Mastery](https://mlsanity.com/paper/2602.09892/) (2026)
- [BeyondSWE: Can Current Code Agent Survive Beyond Single-Repo Bug Fixing?](https://mlsanity.com/paper/2603.03194/) (2026)
- [daVinci-Dev: Agent-native Mid-training for Software Engineering](https://mlsanity.com/paper/2601.18418/) (2026)
- [SWE-MeM: Learning Adaptive Memory Management for Long-Horizon Coding Agents](https://mlsanity.com/paper/2606.28434/) (2026)
- [daVinci-Env: Open SWE Environment Synthesis at Scale](https://mlsanity.com/paper/2603.13023/) (2026)
- [SERA: Soft-Verified Efficient Repository Agents](https://mlsanity.com/paper/2601.20789/) (2026)
- [Open-SWE-Traces: Advancing Dual-Mode Multilingual Distillation for Software Engineering Agents](https://mlsanity.com/paper/2606.16038/) (2026)
- [SWE-Next: Scalable Real-World Software Engineering Tasks for Agents](https://mlsanity.com/paper/2603.20691/) (2026)
- [What Makes Interaction Trajectories Effective for Training Terminal Agents?](https://mlsanity.com/paper/2606.03461/) (2026)
- [Dockerless: Environment-Free Program Verifier for Coding Agents](https://mlsanity.com/paper/2606.28436/) (2026)
- [Orchard: An Open-Source Agentic Modeling Framework](https://mlsanity.com/paper/2605.15040/) (2026)
- [From SWE-ZERO to SWE-HERO: Execution-free to Execution-based Fine-tuning for Software Engineering Agents](https://mlsanity.com/paper/2604.01496/) (2026)
- [SWE-Review: Closing the Loop on Issue Resolution with Agentic Code Review](https://mlsanity.com/paper/2607.06065/) (2026)
- [SWE-Hub: A Unified Production System for Scalable, Executable Software Engineering Tasks](https://mlsanity.com/paper/2603.00575/) (2026)
- [From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents](https://mlsanity.com/paper/2605.21996/) (2026)
- [Mach-Mind-4-Flash Technical Report](https://mlsanity.com/paper/2607.09375/) (2026)
- [Agentic-DPO: From Imitation to Agentic Policy Optimization on Expert Trajectories](https://mlsanity.com/paper/2607.10601/) (2026)
- [FrogNano: Training a 4B Coding Agent via Online Task Synthesis](https://mlsanity.com/paper/2609.07925/) (2026)
- [PhoenixRepair: Rethinking Repair Strategy Exploration in Software Agents](https://mlsanity.com/paper/2607.18859/) (2026)
- [MindForge: Teaching Small Language Models Whole-Life-Cycle Software Engineering via Source-Free Program Synthesis](https://mlsanity.com/paper/2607.27146/) (2026)
- [Unseen-Codebases-Domain Data Synthesis and Training Based on Code Graphs](https://mlsanity.com/paper/2602.20799/) (2026)
- [M2A: Synergizing Mathematical and Agentic Reasoning in Large Language Models](https://mlsanity.com/paper/2605.09879/) (2026)
- [PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks](https://mlsanity.com/paper/2607.28587/) (2026)
- [MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement](https://mlsanity.com/paper/2610.11959/) (2026)
- [Feyospace-v1: How the Cyber Mercury Seven Trained Frontier Cyber Models](https://mlsanity.com/paper/2609.08418/) (2026)
- [One to More, More to One: Category-Aware Iterative Expert Training for Software Engineering Agents](https://mlsanity.com/paper/2609.23377/) (2026)
- [Counterfactual Rollout Replay: Forkable Environments as Free Process Rewards for Software Engineering Agents](https://mlsanity.com/paper/2609.33875/) (2026)
- [Beyond Rule-Based Mutation Testing: Test-Aware Mutant Generation Using Large Language Models](https://mlsanity.com/paper/2609.35841/) (2026)

## Abstract (arXiv)

We present SWE-Lego, a supervised fine-tuning (SFT) recipe designed to achieve state-ofthe-art performance in software engineering (SWE) issue resolving. In contrast to prevalent methods that rely on complex training paradigms (e.g., mid-training, SFT, reinforcement learning, and their combinations), we explore how to push the limits of a lightweight SFT-only approach for SWE tasks. SWE-Lego comprises three core building blocks, with key findings summarized as follows: 1) the SWE-Lego dataset, a collection of 32k highquality task instances and 18k validated trajectories, combining real and synthetic data to complement each other in both quality and quantity; 2) a refined SFT procedure with error masking and a difficulty-based curriculum, which demonstrably improves action quality and overall performance. Empirical results show that with these two building bricks alone,the SFT can push SWE-Lego models to state-of-the-art performance among open-source models of comparable size on SWE-bench Verified: SWE-Lego-Qwen3-8B reaches 42.2%, and SWE-Lego-Qwen3-32B attains 52.6%. 3) We further evaluate and improve test-time scaling (TTS) built upon the SFT foundation. Based on a well-trained verifier, SWE-Lego models can be significantly boosted--for example, 42.2% to 49.6% and 52.6% to 58.8% under TTS@16 for the 8B and 32B models, respectively.