# Test-Driven Development for Code Generation

arXiv: https://arxiv.org/abs/2402.13521 · HTML: https://mlsanity.com/paper/2402.13521/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 42 references that are in the map.
- First posted on arXiv 21 Feb 2024.

## Builds on (most cited first, 18 of 18)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [CodeT: Code Generation with Generated Tests](https://mlsanity.com/paper/2207.10397/) (2022)
- [RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation](https://mlsanity.com/paper/2303.12570/) (2023)
- [LEVER: Learning to Verify Language-to-Code Generation with Execution](https://mlsanity.com/paper/2302.08468/) (2023)
- [Making Large Language Models Better Reasoners with Step-Aware Verifier](https://mlsanity.com/paper/2206.02336/) (2022)
- [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation](https://mlsanity.com/paper/2302.06527/) (2023)
- [CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion](https://mlsanity.com/paper/2310.11248/) (2023)
- [Interactive Code Generation via Test-Driven User-Intent Formalization](https://mlsanity.com/paper/2208.05950/) (2022)
- [Compilable Neural Code Generation with Compiler Feedback](https://mlsanity.com/paper/2203.05132/) (2022)

## Built on by (most cited first, 30 of 34)

- [SkillOps: Managing LLM Agent Skill Libraries as Self-Maintaining Software Ecosystems](https://mlsanity.com/paper/2605.13716/) (2026)
- [Large Language Models (LLMs) for Source Code Analysis: applications, models and datasets](https://mlsanity.com/paper/2503.17502/) (2025)
- [A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System](https://mlsanity.com/paper/2510.09721/) (2025)
- [Automatically Generating Web Applications from Requirements Via Multi-Agent Test-Driven Development](https://mlsanity.com/paper/2509.25297/) (2025)
- [Beyond the Prompt: An Empirical Study of Cursor Rules](https://mlsanity.com/paper/2512.18925/) (2025)
- [Are Coding Agents Generating Over-Mocked Tests? An Empirical Study](https://mlsanity.com/paper/2602.00409/) (2026)
- [TraceCoder: A Trace-Driven Multi-Agent Framework for Automated Debugging of LLM-Generated Code](https://mlsanity.com/paper/2602.06875/) (2026)
- [Curiosity by Design: An LLM-based Coding Assistant Asking Clarification Questions](https://mlsanity.com/paper/2507.21285/) (2025)
- [Understanding and Bridging the Planner-Coder Gap: A Systematic Study on the Robustness of Multi-Agent Systems for Code Generation](https://mlsanity.com/paper/2510.10460/) (2025)
- [CoreCodeBench: Decoupling Code Intelligence via Fine-Grained Repository-Level Tasks](https://mlsanity.com/paper/2507.05281/) (2025)
- [Automated structural testing of LLM-based agents: methods, framework, and case studies](https://mlsanity.com/paper/2601.18827/) (2026)
- [Design choices made by LLM-based test generators prevent them from finding bugs](https://mlsanity.com/paper/2412.14137/) (2024)
- [TENET: One Step Toward Test-Driven Development for Repository-Level Code Generation](https://mlsanity.com/paper/2509.24148/) (2025)
- [Walking the Tightrope of LLMs for Software Development: A Practitioners' Perspective](https://mlsanity.com/paper/2511.06428/) (2025)
- [Code for Machines, Not Just Humans: Quantifying AI-Friendliness with Code Health Metrics](https://mlsanity.com/paper/2601.02200/) (2026)
- [TDD Governance for Multi-Agent Code Generation via Prompt Engineering](https://mlsanity.com/paper/2604.26615/) (2026)
- [Chain of Grounded Objectives: Bridging Process and Goal-oriented Prompting for Code Generation](https://mlsanity.com/paper/2501.13978/) (2025)
- [Understanding and supporting how developers prompt for LLM-powered code editing in practice](https://mlsanity.com/paper/2504.20196/) (2025)
- [Breaking Memorization Barriers in LLM Code Fine-Tuning via Information Bottleneck for Improved Generalization](https://mlsanity.com/paper/2510.16022/) (2025)
- [OOM-RL: Out-of-Money Reinforcement Learning Market-Driven Alignment for LLM-Based Multi-Agent Systems](https://mlsanity.com/paper/2604.11477/) (2026)
- [SemaDiff: Identifying Semantic-Changing Commits with Generated Code and Tests](https://mlsanity.com/paper/2607.13111/) (2026)
- [Experimental Analysis of Productive Interaction Strategy with ChatGPT: User Study on Function and Project-level Code Generation Tasks](https://mlsanity.com/paper/2508.04125/) (2025)
- [Leveraging Test Driven Development with Large Language Models for Reliable and Verifiable Spreadsheet Code Generation: A Research Framework](https://mlsanity.com/paper/2510.15585/) (2025)
- [A Comparison of Conversational Models and Humans in Answering Technical Questions: the Firefox Case](https://mlsanity.com/paper/2510.21933/) (2025)
- [NALA_MAINZ at BLP-2025 Task 2: A Multi-agent Approach for Bangla Instruction to Python Code Generation](https://mlsanity.com/paper/2511.16787/) (2025)
- [Chain of Unit-Physics: A Primitive-Centric Approach to Scientific Code Synthesis](https://mlsanity.com/paper/2512.01010/) (2025)
- [Advancing Language Models for Code-related Tasks](https://mlsanity.com/paper/2601.04526/) (2026)
- [Inferring Code Correctness from Specification](https://mlsanity.com/paper/2605.29822/) (2026)
- [An Exploratory Case Study of LLM-Assisted Refactoring and Gameplay Feature Generation in an Endless Runner Game](https://mlsanity.com/paper/2606.21171/) (2026)
- [Prompt Coach: An Empirical Evaluation of an Agentic Tutor for Learning Prompt Engineering in Software Development](https://mlsanity.com/paper/2607.06074/) (2026)

## Abstract (arXiv)

Recent Large Language Models (LLMs) have demonstrated significant capabilities in generating code snippets directly from problem statements. This increasingly automated process mirrors traditional human-led software development, where code is often written in response to a requirement. Historically, Test-Driven Development (TDD) has proven its merit, requiring developers to write tests before the functional code, ensuring alignment with the initial problem statements. Applying TDD principles to LLM-based code generation offers one distinct benefit: it enables developers to verify the correctness of generated code against predefined tests. This paper investigates if and how TDD can be incorporated into AI-assisted code-generation processes. We experimentally evaluate our hypothesis that providing LLMs like GPT-4 and Llama 3 with tests in addition to the problem statements enhances code generation outcomes. We experimented with established function-level code generation benchmarks such as MBPP and HumanEval. Our results consistently demonstrate that including test cases leads to higher success in solving programming challenges. We assert that TDD is a promising paradigm for helping ensure that the code generated by LLMs effectively captures the requirements.