mlsanityData to 14 Oct 2026

SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Manner

arXiv 2506.09003 ↗First posted 10 Jun 2025cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

We introduce **SWE-Flow**, a novel data synthesis framework grounded in Test-Driven Development (TDD). Unlike existing software engineering data that rely on human-submitted issues, **SWE-Flow** automatically infers incremental development steps directly from unit tests, which inherently encapsulate high-level requirements. The core of **SWE-Flow** is the construction of a Runtime Dependency Graph (RDG), which precisely captures function interactions, enabling the generation of a structured, step-by-step *development schedule*. At each step, **SWE-Flow** produces a partial codebase, the corresponding unit tests, and the necessary code modifications, resulting in fully verifiable TDD tasks. With this approach, we generated 16,061 training instances and 2,020 test instances from real-world GitHub projects, creating the **SWE-Flow-Eval** benchmark. Our experiments show that fine-tuning open model on this dataset significantly improves performance in TDD-based coding. To facilitate further research, we release all code, datasets, models, and Docker images at [Github](https://github.com/Hambaobao/SWE-Flow).

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 44 of its 58 references that are in the map. First posted on arXiv 10 Jun 2025.

Builds on

44 of 58 references.

References · 44

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  6. 2021Program Synthesis with Large Language Modelscited by 3,637
  7. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  9. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  10. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  11. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  12. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
32 more
  1. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  2. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  3. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  4. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  5. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
  6. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  7. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 744
  8. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  9. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 654
  10. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 546
  11. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 502
  12. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 452
  13. 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 367
  14. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 336
  15. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 333
  16. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 300
  17. 2022Efficient Training of Language Models to Fill in the Middlecited by 255
  18. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 130
  19. 2024TableBench: A Comprehensive and Complex Benchmark for Table Question Answeringcited by 128
  20. 2024DebugBench: Evaluating Debugging Capability of Large Language Modelscited by 76
  21. 2021CoSQA: 20,000+ Web Queries for Code Search and Question Answeringcited by 75
  22. 2024TableGPT2: A Large Multimodal Model with Tabular Data Integrationcited by 72
  23. 2024AutoKaggle: A Multi-Agent Framework for Autonomous Data Science Competitionscited by 70
  24. 2024FullStack Bench: Evaluating LLMs as Full Stack Coderscited by 58
  25. 2023CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translationcited by 57
  26. 2024Commit0: Library Generation from Scratchcited by 52
  27. 2024McEval: Massively Multilingual Code Evaluationcited by 44
  28. 2024Evaluating and Aligning CodeLLMs on Human Preferencecited by 22
  29. 2024M2rc-Eval: Massively Multilingual Repository-level Code Completion Evaluationcited by 21
  30. 2024ExecRepoBench: Multi-level Executable Code Completion Evaluationcited by 20
  31. 2024MdEval: Massively Multilingual Code Debuggingcited by 16
  32. 2025DI-BENCH: Benchmarking Large Language Models on Dependency Inference with Testable Repositories at Scalecited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 16.

20 papers.

Sort:

By area:

  1. 2026Qwen3-Coder-Next Technical Reportcited by 71
  2. 2026FeatureBench: Benchmarking Agentic Coding for Complex Feature Developmentcited by 34
  3. 2026SWE-rebench V2: Language-Agnostic SWE Task Collection at Scalecited by 29
  4. 2025SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositoriescited by 28
  5. 2026SWE-Universe: Scale Real-World Verifiable Environments to Millionscited by 19
  6. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 15
  7. 2025Training Versatile Coding Agents in Synthetic Environmentscited by 13
  8. 2026MegaFlow: Large-Scale Distributed Orchestration System for the Agentic Eracited by 9
  9. 2025SWE-Factory: Your Automated Factory for Issue Resolution Training Data and Evaluation Benchmarkscited by 7
  10. 2026From Completion to Editing: Unlocking Context-Aware Code Infilling via Search-and-Replace Instruction Tuningcited by 6
  11. 2026MEnvAgent: Scalable Polyglot Environment Construction for Verifiable Software Engineeringcited by 4
  12. 2026RealChart2Code: Advancing Chart-to-Code Generation with Real Data and Multi-Task Evaluationcited by 4
8 more
  1. 2026FrogNano: Training a 4B Coding Agent via Online Task Synthesiscited by 4
  2. 2026Breaking the Solver Bottleneck: Training Task Generators at the Learnable Frontiercited by 3
  3. 2026CodeMidas: Scaling Agentic Coding RL Environments from Code Itselfcited by 2
  4. 2026PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarkscited by 1
  5. 2026Feyospace-v1: How the Cyber Mercury Seven Trained Frontier Cyber Modelscited by 0
  6. 2026Test-Driven Approaches to Software Engineering with Large Language Models: A Survey of Phases, Tasks, and Agent Skillscited by 0
  7. 2026SWE-Journey: Towards More Realistic Evaluation of Coding Assistants through Long-Horizon, Multi-Turn Interactioncited by 0
  8. 2026Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewardscited by 0