mlsanityData to 14 Oct 2026

WIP: Assessing the Effectiveness of ChatGPT in Preparatory Testing Activities

arXiv 2503.03951 ↗First posted 5 Mar 2025cs.SE cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

This innovative practice WIP paper describes a research study that explores the integration of ChatGPT into the software testing curriculum and evaluates its effectiveness compared to human-generated testing artifacts. In a Capstone Project course, students were tasked with generating preparatory testing artifacts using ChatGPT prompts, which they had previously created manually. Their understanding and the effectiveness of the Artificial Intelligence generated artifacts were assessed through targeted questions. The results, drawn from this in-class assignment at a North American community college indicate that while ChatGPT can automate many testing preparation tasks, it cannot fully replace human expertise. However, students, already familiar with Information Technology at the postgraduate level, found the integration of ChatGPT into their workflow to be straightforward. The study suggests that AI can be gradually introduced into software testing education to keep pace with technological advancements.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 9 of its 26 references that are in the map. First posted on arXiv 5 Mar 2025.

Builds on

9 of 26 references.

References · 9

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Using an LLM to Help With Code Understandingcited by 228
  3. 2023An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generationcited by 194
  4. 2023Software Testing with Large Language Models: Survey, Landscape, and Visioncited by 166
  5. 2023InferFix: End-to-End Program Repair with LLMscited by 114
  6. 2022Large Language Models are Few-shot Testers: Exploring LLM-based General Bug Reproductioncited by 111
  7. 2023The Robots are Here: Navigating the Generative AI Revolution in Computing Educationcited by 86
  8. 2023Make LLM a Testing Expert: Bringing Human-like Interaction to Mobile GUI Testing via Functionality-aware Decisionscited by 41
  9. 2023GPT-3.5, GPT-4, or BARD? Evaluating LLMs Reasoning Ability in Zero-Shot Setting and Performance Boosting Through Promptscited by 34

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Navigating the growing field of research on AI for software testing -- the taxonomy for AI-augmented software testing and an ontology-driven literature surveycited by 2