# Challenges and guidelines on designing test cases for test bots

arXiv: https://arxiv.org/abs/2004.10143 · HTML: https://mlsanity.com/paper/2004.10143/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Apr 2020.

## Built on by (most cited first, 2 of 2)

- [Motivations, Challenges, Best Practices, and Benefits for Bots and Conversational Agents in Software Engineering: A Multivocal Literature Review](https://mlsanity.com/paper/2409.11864/) (2024)
- [DevBots can co-design APIs](https://mlsanity.com/paper/2312.05733/) (2023)

## Abstract (arXiv)

Test bots are automated testing tools that autonomously and periodically run a set of test cases that check whether the system under test meets the requirements set forth by the customer. The automation decreases the amount of time a development team spends on testing. As development projects become larger, it is important to focus on improving the test bots by designing more effective test cases because otherwise time and usage costs can increase greatly and misleading conclusions from test results might be drawn, such as false positives in the test execution. However, literature currently lacks insights on how test case design affects the effectiveness of test bots. This paper uses a case study approach to investigate those effects by identifying challenges in designing tests for test bots. Our results include guidelines for test design schema for such bots that support practitioners in overcoming the challenges mentioned by participants during our study.