Synthesizing File-Level Data for Unit Test Generation with Chain-of-Thoughts via Self-Debugging
On the map: LLM reasoning → Reasoning for agents, planning and code
Abstract (arXiv)
Automatic unit test (UT) generation is essential for software quality assurance, but existing approaches--including symbolic execution, search-based approaches, and recent LLM-based generators--struggle to produce human-quality tests with correct, meaningful assertions and reliable chain-of-thought (CoT) explanations. We identify a gap in UT training data: repository-mined tests lack developer CoTs, while LLM-distilled CoTs are often incorrect or incomplete. To address this issue, we propose a novel data-distillation approach that uses self-debugging to produce high-quality UT training examples paired with faithful CoTs. Our approach combines (1) guided test repair, a heuristic loop (error-, failure-, and coverage-focused steps) that asks the used model to diagnose and iteratively fix generated tests, and (2) CoT compression, which compacts original and debugging CoTs into concise explanations that directly justify correct tests. We apply this pipeline to a large corpus of open-source projects to construct a dataset of 74,518 high-quality <focal method, test, CoT> examples, and then use it for supervised fine-tuning of a base model. An empirical evaluation shows that the fine-tuned model achieves high UT generation effectiveness: it attains a pass rate of 36.17% on test assertions, a branch coverage of 43.90%, and a mutation score of 88.66%, substantially higher than state-of-the-art commercial models like o4-mini.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 24 of its 58 references that are in the map. First posted on arXiv 3 Feb 2026.
Builds on
24 of 58 references.
Closest ancestors · 6
Same direction, newest first
- 2025OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critique11 Jul
- 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Coding2 Apr
- 2025Uncertainty-Guided Chain-of-Thought for Code Generation with LLMs19 Mar
- 2025Dynamic Scaling of Unit Tests for Code Reward Modeling2 Jan
- 2024Qwen2.5-Coder Technical Report18 Sep
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code12 Mar
Other references · 18
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023Textbooks Are All You Needcited by 561
- 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
- 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
- 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
- 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
- 2022Formal Mathematics Statement Curriculum Learningcited by 141
- 2020Unit Test Case Generation with Transformers and Focal Contextcited by 118
- 2022Automating Code Review Activities by Large-Scale Pre-trainingcited by 98
- 2024TestGenEval: A Real World Unit Test Generation and Test Completion Benchmarkcited by 49
6 more
- 2025Seed-Coder: Let the Code Model Curate Data for Itselfcited by 46
- 2024On the Evaluation of Large Language Models in Unit Test Generationcited by 33
- 2024The Imperative of Conversation Analysis in the Era of LLMs: A Survey of Tasks, Techniques, and Trendscited by 11
- 2025Teaching Your Models to Understand Code via Focal Preference Alignmentcited by 10
- 2025Automated Generation of Issue-Reproducing Tests by Combining LLMs and Search-Based Testingcited by 5
- 2025Clarifying Semantics of In-Context Examples for Unit Test Generationcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Learning to Generate Unit Tests for Automated Debugging
- 2024Large-scale, Independent and Comprehensive study of the power of LLMs for test case generation
- 2025Go-UT-Bench: A Fine-Tuning Dataset for LLM-Based Unit Test Generation in Go
- 2026VibeCheck: Assessing the Quality of LLM-Generated Unit Tests: A Multi-agent Empirical Study across Heterogeneous Repositories
- 2026TAM-Eval: Evaluating LLMs for Automated Unit Test Maintenance
- 2025AugmenTest: Enhancing Tests with LLM-Driven Oracles
- 2026On the risk of coding before testing: An empirical study on LLM-based test generation workflow
- 2026LLM vs. Human Unit Tests: Fault Detection on Real Python Bugs