# Automated Unit Test Generation for Python

arXiv: https://arxiv.org/abs/2007.14049 · HTML: https://mlsanity.com/paper/2007.14049/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jul 2020.

## Built on by (most cited first, 18 of 18)

- [Execution-Based Evaluation for Open-Domain Code Generation](https://mlsanity.com/paper/2212.10481/) (2022)
- [Code-Aware Prompting: A study of Coverage Guided Test Generation in Regression Setting using LLM](https://mlsanity.com/paper/2402.00097/) (2024)
- [Knowledge Transfer from High-Resource to Low-Resource Programming Languages for Code LLMs](https://mlsanity.com/paper/2308.09895/) (2023)
- [Data-driven Discovery with Large Generative Models](https://mlsanity.com/paper/2402.13610/) (2024)
- [LLM-Powered Test Case Generation for Detecting Bugs in Plausible Programs](https://mlsanity.com/paper/2404.10304/) (2024)
- [Two is Better Than One: Digital Siblings to Improve Autonomous Driving Testing](https://mlsanity.com/paper/2305.08060/) (2023)
- [LLM Test Generation via Iterative Hybrid Program Analysis](https://mlsanity.com/paper/2503.13580/) (2025)
- [LExecutor: Learning-Guided Execution](https://mlsanity.com/paper/2302.02343/) (2023)
- [MultiFileTest: A Multi-File-Level LLM Unit Test Generation Benchmark and Impact of Error Fixing Mechanisms](https://mlsanity.com/paper/2502.06556/) (2025)
- [Semi-Instruct: Bridging Natural-Instruct and Self-Instruct for Code Large Language Models](https://mlsanity.com/paper/2403.00338/) (2024)
- [CSEPrompts: A Benchmark of Introductory Computer Science Prompts](https://mlsanity.com/paper/2404.02540/) (2024)
- [Treefix: Enabling Execution with a Tree of Prefixes](https://mlsanity.com/paper/2501.12339/) (2025)
- [GitChameleon 2.0: Evaluating AI Code Generation Against Python Library Version Incompatibilities](https://mlsanity.com/paper/2507.12367/) (2025)
- [Automated Unit Test Case Generation: A Systematic Literature Review](https://mlsanity.com/paper/2504.20357/) (2025)
- [SCATE: Learning to Supervise Coding Agents for Cost-Effective Test Generation](https://mlsanity.com/paper/2607.08983/) (2026)
- [AgentExecutor: Partial Code Execution via Agentic Context Generation](https://mlsanity.com/paper/2608.05959/) (2026)
- [Automated Support for Unit Test Generation: A Tutorial Book Chapter](https://mlsanity.com/paper/2110.13575/) (2021)
- [Beyond Accuracy: An Empirical Study on Unit Testing in Open-source Deep Learning Projects](https://mlsanity.com/paper/2402.16546/) (2024)

## Abstract (arXiv)

Automated unit test generation is an established research field, and mature test generation tools exist for statically typed programming languages such as Java. It is, however, substantially more difficult to automatically generate supportive tests for dynamically typed programming languages such as Python, due to the lack of type information and the dynamic nature of the language. In this paper, we describe a foray into the problem of unit test generation for dynamically typed languages. We introduce Pynguin, an automated unit test generation framework for Python. Using Pynguin, we aim to empirically shed light on two central questions: (1) Do well-established search-based test generation methods, previously evaluated only on statically typed languages, generalise to dynamically typed languages? (2) What is the influence of incomplete type information and dynamic typing on the problem of automated test generation? Our experiments confirm that evolutionary algorithms can outperform random test generation also in the context of Python, and can even alleviate the problem of absent type information to some degree. However, our results demonstrate that dynamic typing nevertheless poses a fundamental issue for test generation, suggesting future work on integrating type inference.