# Uncertainty-Driven Black-Box Test Data Generation

arXiv: https://arxiv.org/abs/1608.03181 · HTML: https://mlsanity.com/paper/1608.03181/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Aug 2016.

## Built on by (most cited first, 4 of 4)

- [Constrained Adversarial Learning for Automated Software Testing: a literature review](https://mlsanity.com/paper/2303.07546/) (2023)
- [The Integration of Machine Learning into Automated Test Generation: A Systematic Mapping Study](https://mlsanity.com/paper/2206.10210/) (2022)
- [Automated Support for Unit Test Generation: A Tutorial Book Chapter](https://mlsanity.com/paper/2110.13575/) (2021)
- [Learning-Based Testing for Deep Learning: Enhancing Model Robustness with Adversarial Input Prioritization](https://mlsanity.com/paper/2509.23961/) (2025)

## Abstract (arXiv)

We can never be certain that a software system is correct simply by testing it, but with every additional successful test we become less uncertain about its correctness. In absence of source code or elaborate specifications and models, tests are usually generated or chosen randomly. However, rather than randomly choosing tests, it would be preferable to choose those tests that decrease our uncertainty about correctness the most. In order to guide test generation, we apply what is referred to in Machine Learning as "Query Strategy Framework": We infer a behavioural model of the system under test and select those tests which the inferred model is "least certain" about. Running these tests on the system under test thus directly targets those parts about which tests so far have failed to inform the model. We provide an implementation that uses a genetic programming engine for model inference in order to enable an uncertainty sampling technique known as "query by committee", and evaluate it on eight subject systems from the Apache Commons Math framework and JodaTime. The results indicate that test generation using uncertainty sampling outperforms conventional and Adaptive Random Testing.