SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Manner
Not on the map. Why?
Abstract (arXiv)
We introduce **SWE-Flow**, a novel data synthesis framework grounded in Test-Driven Development (TDD). Unlike existing software engineering data that rely on human-submitted issues, **SWE-Flow** automatically infers incremental development steps directly from unit tests, which inherently encapsulate high-level requirements. The core of **SWE-Flow** is the construction of a Runtime Dependency Graph (RDG), which precisely captures function interactions, enabling the generation of a structured, step-by-step *development schedule*. At each step, **SWE-Flow** produces a partial codebase, the corresponding unit tests, and the necessary code modifications, resulting in fully verifiable TDD tasks. With this approach, we generated 16,061 training instances and 2,020 test instances from real-world GitHub projects, creating the **SWE-Flow-Eval** benchmark. Our experiments show that fine-tuning open model on this dataset significantly improves performance in TDD-based coding. To facilitate further research, we release all code, datasets, models, and Docker images at [Github](https://github.com/Hambaobao/SWE-Flow).
Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 44 of its 58 references that are in the map. First posted on arXiv 10 Jun 2025.
Builds on
44 of 58 references.
References · 44
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2024GPT-4o System Cardcited by 4,588
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
- 2023Code Llama: Open Foundation Models for Codecited by 2,139
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
32 more
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
- 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
- 2024Qwen2.5-Coder Technical Reportcited by 1,203
- 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
- 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
- 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 744
- 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
- 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 654
- 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 546
- 2024StarCoder 2 and The Stack v2: The Next Generationcited by 502
- 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 452
- 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 367
- 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 336
- 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 333
- 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 300
- 2022Efficient Training of Language Models to Fill in the Middlecited by 255
- 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 130
- 2024TableBench: A Comprehensive and Complex Benchmark for Table Question Answeringcited by 128
- 2024DebugBench: Evaluating Debugging Capability of Large Language Modelscited by 76
- 2021CoSQA: 20,000+ Web Queries for Code Search and Question Answeringcited by 75
- 2024TableGPT2: A Large Multimodal Model with Tabular Data Integrationcited by 72
- 2024AutoKaggle: A Multi-Agent Framework for Autonomous Data Science Competitionscited by 70
- 2024FullStack Bench: Evaluating LLMs as Full Stack Coderscited by 58
- 2023CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translationcited by 57
- 2024Commit0: Library Generation from Scratchcited by 52
- 2024McEval: Massively Multilingual Code Evaluationcited by 44
- 2024Evaluating and Aligning CodeLLMs on Human Preferencecited by 22
- 2024M2rc-Eval: Massively Multilingual Repository-level Code Completion Evaluationcited by 21
- 2024ExecRepoBench: Multi-level Executable Code Completion Evaluationcited by 20
- 2024MdEval: Massively Multilingual Code Debuggingcited by 16
- 2025DI-BENCH: Benchmarking Large Language Models on Dependency Inference with Testable Repositories at Scalecited by 7
Built on by
Citing papers per year: 2025: 4; 2026: 16.
20 papers.
Sort:
By area:
- 2026Qwen3-Coder-Next Technical Reportcited by 71
- 2026FeatureBench: Benchmarking Agentic Coding for Complex Feature Developmentcited by 34
- 2026SWE-rebench V2: Language-Agnostic SWE Task Collection at Scalecited by 29
- 2025SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositoriescited by 28
- 2026SWE-Universe: Scale Real-World Verifiable Environments to Millionscited by 19
- 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 15
- 2025Training Versatile Coding Agents in Synthetic Environmentscited by 13
- 2026MegaFlow: Large-Scale Distributed Orchestration System for the Agentic Eracited by 9
- 2025SWE-Factory: Your Automated Factory for Issue Resolution Training Data and Evaluation Benchmarkscited by 7
- 2026From Completion to Editing: Unlocking Context-Aware Code Infilling via Search-and-Replace Instruction Tuningcited by 6
- 2026MEnvAgent: Scalable Polyglot Environment Construction for Verifiable Software Engineeringcited by 4
- 2026RealChart2Code: Advancing Chart-to-Code Generation with Real Data and Multi-Task Evaluationcited by 4
8 more
- 2026FrogNano: Training a 4B Coding Agent via Online Task Synthesiscited by 4
- 2026Breaking the Solver Bottleneck: Training Task Generators at the Learnable Frontiercited by 3
- 2026CodeMidas: Scaling Agentic Coding RL Environments from Code Itselfcited by 2
- 2026PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarkscited by 1
- 2026Feyospace-v1: How the Cyber Mercury Seven Trained Frontier Cyber Modelscited by 0
- 2026Test-Driven Approaches to Software Engineering with Large Language Models: A Survey of Phases, Tasks, and Agent Skillscited by 0
- 2026SWE-Journey: Towards More Realistic Evaluation of Coding Assistants through Long-Horizon, Multi-Turn Interactioncited by 0
- 2026Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewardscited by 0
- 2026Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards13 Oct 2026
- 2026SWE-Journey: Towards More Realistic Evaluation of Coding Assistants through Long-Horizon, Multi-Turn Interaction8 Oct 2026
- 2026CodeMidas: Scaling Agentic Coding RL Environments from Code Itself18 Sep 2026
- 2026Test-Driven Approaches to Software Engineering with Large Language Models: A Survey of Phases, Tasks, and Agent Skills10 Sep 2026
- 2026Feyospace-v1: How the Cyber Mercury Seven Trained Frontier Cyber Models8 Sep 2026
- 2026FrogNano: Training a 4B Coding Agent via Online Task Synthesis7 Sep 2026
- 2026PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks30 Jul 2026
- 2026Breaking the Solver Bottleneck: Training Task Generators at the Learnable Frontier10 Jun 2026
- 2026RealChart2Code: Advancing Chart-to-Code Generation with Real Data and Multi-Task Evaluation26 Mar 2026
- 2026Qwen3-Coder-Next Technical Report28 Feb 2026
- 2026SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale27 Feb 2026
- 2026FeatureBench: Benchmarking Agentic Coding for Complex Feature Development11 Feb 2026
8 more
- 2026SWE-Universe: Scale Real-World Verifiable Environments to Millions2 Feb 2026
- 2026MEnvAgent: Scalable Polyglot Environment Construction for Verifiable Software Engineering30 Jan 2026
- 2026From Completion to Editing: Unlocking Context-Aware Code Infilling via Search-and-Replace Instruction Tuning19 Jan 2026
- 2026MegaFlow: Large-Scale Distributed Orchestration System for the Agentic Era12 Jan 2026
- 2025SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositories19 Dec 2025
- 2025Training Versatile Coding Agents in Synthetic Environments13 Dec 2025
- 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence23 Nov 2025
- 2025SWE-Factory: Your Automated Factory for Issue Resolution Training Data and Evaluation Benchmarks12 Jun 2025
- 2026Qwen3-Coder-Next Technical Reportcited by 71
- 2026FeatureBench: Benchmarking Agentic Coding for Complex Feature Developmentcited by 34
- 2026SWE-rebench V2: Language-Agnostic SWE Task Collection at Scalecited by 29
- 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 15
- 2025Training Versatile Coding Agents in Synthetic Environmentscited by 13
- 2026MegaFlow: Large-Scale Distributed Orchestration System for the Agentic Eracited by 9
- 2026FrogNano: Training a 4B Coding Agent via Online Task Synthesiscited by 4
- 2026CodeMidas: Scaling Agentic Coding RL Environments from Code Itselfcited by 2
- 2026Test-Driven Approaches to Software Engineering with Large Language Models: A Survey of Phases, Tasks, and Agent Skillscited by 0
- 2026SWE-Journey: Towards More Realistic Evaluation of Coding Assistants through Long-Horizon, Multi-Turn Interactioncited by 0
- 2026Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewardscited by 0
- 2026Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards13 Oct 2026
- 2026SWE-Journey: Towards More Realistic Evaluation of Coding Assistants through Long-Horizon, Multi-Turn Interaction8 Oct 2026
- 2026CodeMidas: Scaling Agentic Coding RL Environments from Code Itself18 Sep 2026
- 2026Test-Driven Approaches to Software Engineering with Large Language Models: A Survey of Phases, Tasks, and Agent Skills10 Sep 2026
- 2026FrogNano: Training a 4B Coding Agent via Online Task Synthesis7 Sep 2026
- 2026Qwen3-Coder-Next Technical Report28 Feb 2026
- 2026SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale27 Feb 2026
- 2026FeatureBench: Benchmarking Agentic Coding for Complex Feature Development11 Feb 2026
- 2026MegaFlow: Large-Scale Distributed Orchestration System for the Agentic Era12 Jan 2026
- 2025Training Versatile Coding Agents in Synthetic Environments13 Dec 2025
- 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence23 Nov 2025
- 2025SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositoriescited by 28
- 2026SWE-Universe: Scale Real-World Verifiable Environments to Millionscited by 19
- 2025SWE-Factory: Your Automated Factory for Issue Resolution Training Data and Evaluation Benchmarkscited by 7
- 2026From Completion to Editing: Unlocking Context-Aware Code Infilling via Search-and-Replace Instruction Tuningcited by 6
- 2026MEnvAgent: Scalable Polyglot Environment Construction for Verifiable Software Engineeringcited by 4
- 2026PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarkscited by 1
- 2026Feyospace-v1: How the Cyber Mercury Seven Trained Frontier Cyber Modelscited by 0
- 2026Feyospace-v1: How the Cyber Mercury Seven Trained Frontier Cyber Models8 Sep 2026
- 2026PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks30 Jul 2026
- 2026SWE-Universe: Scale Real-World Verifiable Environments to Millions2 Feb 2026
- 2026MEnvAgent: Scalable Polyglot Environment Construction for Verifiable Software Engineering30 Jan 2026
- 2026From Completion to Editing: Unlocking Context-Aware Code Infilling via Search-and-Replace Instruction Tuning19 Jan 2026
- 2025SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositories19 Dec 2025
- 2025SWE-Factory: Your Automated Factory for Issue Resolution Training Data and Evaluation Benchmarks12 Jun 2025