# Generative AI to Generate Test Data Generators

arXiv: https://arxiv.org/abs/2401.17626 · HTML: https://mlsanity.com/paper/2401.17626/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 20 references that are in the map.
- First posted on arXiv 31 Jan 2024.

## Builds on (most cited first, 6 of 6)

- [Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study](https://mlsanity.com/paper/2303.17466/) (2023)
- [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation](https://mlsanity.com/paper/2302.06527/) (2023)
- [Synchromesh: Reliable code generation from pre-trained language models](https://mlsanity.com/paper/2201.11227/) (2022)
- [Having Beer after Prayer? Measuring Cultural Bias in Large Language Models](https://mlsanity.com/paper/2305.14456/) (2023)
- [An Empirical Study of the Non-determinism of ChatGPT in Code Generation](https://mlsanity.com/paper/2308.02828/) (2023)
- [Fill in the Blank: Context-aware Automated Text Input Generation for Mobile GUI Testing](https://mlsanity.com/paper/2212.04732/) (2022)

## Built on by (most cited first, 4 of 4)

- [CoSQA+: Pioneering the Multi-Choice Code Search Benchmark with Test-Driven Agents](https://mlsanity.com/paper/2406.11589/) (2024)
- [Leveraging Large Language Models for Enhancing the Understandability of Generated Unit Tests](https://mlsanity.com/paper/2408.11710/) (2024)
- [Towards Iterative End-to-End Software Development: A Feature-Driven Multi-Agent Framework](https://mlsanity.com/paper/2511.02399/) (2025)
- [Unify and Triumph: Polyglot, Diverse, and Self-Consistent Generation of Unit Tests with LLMs](https://mlsanity.com/paper/2503.16144/) (2025)

## Abstract (arXiv)

Generating fake data is an essential dimension of modern software testing, as demonstrated by the number and significance of data faking libraries. Yet, developers of faking libraries cannot keep up with the wide range of data to be generated for different natural languages and domains. In this paper, we assess the ability of generative AI for generating test data in different domains. We design three types of prompts for Large Language Models (LLMs), which perform test data generation tasks at different levels of integrability: 1) raw test data generation, 2) synthesizing programs in a specific language that generate useful test data, and 3) producing programs that use state-of-the-art faker libraries. We evaluate our approach by prompting LLMs to generate test data for 11 domains. The results show that LLMs can successfully generate realistic test data generators in a wide range of domains at all three levels of integrability.