# An Empirical Study of Automated Unit Test Generation for Python

arXiv: https://arxiv.org/abs/2111.05003 · HTML: https://mlsanity.com/paper/2111.05003/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Nov 2021.

## Built on by (most cited first, 10 of 10)

- [CoverUp: Effective High Coverage Test Generation for Python](https://mlsanity.com/paper/2403.16218/) (2024)
- [On the Impacts of Contexts on Repository-Level Code Generation](https://mlsanity.com/paper/2406.11927/) (2024)
- [Unit Test Generation using Generative AI : A Comparative Performance Analysis of Autogeneration Tools](https://mlsanity.com/paper/2312.10622/) (2023)
- [The Program Testing Ability of Large Language Models for Code](https://mlsanity.com/paper/2310.05727/) (2023)
- [Automating the Correctness Assessment of AI-generated Code for Security Contexts](https://mlsanity.com/paper/2310.18834/) (2023)
- [Navigating the Labyrinth: Path-Sensitive Unit Test Generation with Large Language Models](https://mlsanity.com/paper/2509.23812/) (2025)
- [Generative Language Models Potential for Requirement Engineering Applications: Insights into Current Strengths and Limitations](https://mlsanity.com/paper/2412.00959/) (2024)
- [Unify and Triumph: Polyglot, Diverse, and Self-Consistent Generation of Unit Tests with LLMs](https://mlsanity.com/paper/2503.16144/) (2025)
- [CUBETESTERAI: Automated JUnit Test Generation using the LLaMA Model](https://mlsanity.com/paper/2504.15286/) (2025)
- [Can LLMs Test Terminal User Interfaces?](https://mlsanity.com/paper/2608.03743/) (2026)

## Abstract (arXiv)

Various mature automated test generation tools exist for statically typed programming languages such as Java. Automatically generating unit tests for dynamically typed programming languages such as Python, however, is substantially more difficult due to the dynamic nature of these languages as well as the lack of type information. Our Pynguin framework provides automated unit test generation for Python. In this paper, we extend our previous work on Pynguin to support more aspects of the Python language, and by studying a larger variety of well-established state of the art test-generation algorithms, namely DynaMOSA, MIO, and MOSA. Furthermore, we improved our Pynguin tool to generate regression assertions, whose quality we also evaluate. Our experiments confirm that evolutionary algorithms can outperform random test generation also in the context of Python, and similar to the Java world, DynaMOSA yields the highest coverage results. However, our results also demonstrate that there are still fundamental remaining issues, such as inferring type information for code without this information, currently limiting the effectiveness of test generation for Python.