mlsanityData to 14 Oct 2026

Leveraging Large Language Models for Enhancing the Understandability of Generated Unit Tests

arXiv 2408.11710 ↗First posted 21 Aug 2024cs.SE cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Automated unit test generators, particularly search-based software testing tools like EvoSuite, are capable of generating tests with high coverage. Although these generators alleviate the burden of writing unit tests, they often pose challenges for software engineers in terms of understanding the generated tests. To address this, we introduce UTGen, which combines search-based software testing and large language models to enhance the understandability of automatically generated test cases. We achieve this enhancement through contextualizing test data, improving identifier naming, and adding descriptive comments. Through a controlled experiment with 32 participants from both academia and industry, we investigate how the understandability of unit tests affects a software engineer's ability to perform bug-fixing tasks. We selected bug-fixing to simulate a real-world scenario that emphasizes the importance of understandable test cases. We observe that participants working on assignments with UTGen test cases fix up to 33% more bugs and use up to 20% less time when compared to baseline test cases. From the post-test questionnaire, we gathered that participants found that enhanced test names, test data, and variable names improved their bug-fixing process.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 29 of its 76 references that are in the map. First posted on arXiv 21 Aug 2024.

Builds on

29 of 76 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  6. 2023StarCoder: may the source be with you!cited by 799
  7. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 479
  8. 2023An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generationcited by 194
  9. 2023Large Language Models for Software Engineering: Survey and Open Problemscited by 181
  10. 2023Software Testing with Large Language Models: Survey, Landscape, and Visioncited by 166
  11. 2021Retrieval Augmented Code Generation and Summarizationcited by 149
  12. 2023Structured Chain-of-Thought Prompting for Code Generationcited by 138
17 more
  1. 2020Retrieval-Augmented Generation for Code Summarization via Hybrid GNNcited by 89
  2. 2024Automated Unit Test Improvement using Large Language Models at Metacited by 75
  3. 2021Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Taskscited by 70
  4. 2023Using Large Language Models to Generate JUnit Tests: An Empirical Studycited by 53
  5. 2023Effective Test Generation Using Pre-trained Large Language Models and Mutation Testingcited by 51
  6. 2022CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequencescited by 50
  7. 2024Language Models for Code Completion: A Practical Evaluationcited by 39
  8. 2023Fully Autonomous Programming with Large Language Modelscited by 31
  9. 2023CAT-LM: Training Language Models on Aligned Code And Testscited by 28
  10. 2023Extending Source Code Pre-Trained Language Models to Summarise Decompiled Binariescited by 26
  11. 2023Reinforcement Learning from Automatic Feedback for High-Quality Unit Test Generationcited by 21
  12. 2023LLM for Test Script Generation and Migration: Challenges, Capabilities, and Opportunitiescited by 19
  13. 2019An Experience Report On Applying Software Testing Academic Results In Industry: We Need Usable Automated Test Generationcited by 4
  14. 2024Generative AI to Generate Test Data Generatorscited by 4
  15. 2021How Developers Engineer Test Cases: An Observational Studycited by 3
  16. 2023State-Of-The-Practice in Quality Assurance in Java-Based Open Source Software Developmentcited by 2
  17. 2020Generating Class-Level Integration Tests Using Call Site Informationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 3.

11 papers.

  1. 2025LLM Test Generation via Iterative Hybrid Program Analysiscited by 9
  2. 2026Test vs Mutant: Adversarial LLM Agents for Robust Unit Test Generationcited by 7
  3. 2025Using LLMs in Generating Design Rationale for Software Architecture Decisionscited by 4
  4. 2025Clarifying Semantics of In-Context Examples for Unit Test Generationcited by 3
  5. 2025A Multi-agent Onboarding Assistant based on Large Language Models, Retrieval Augmented Generation, and Chain-of-Thoughtcited by 2
  6. 2025Prompt-with-Me: in-IDE Structured Prompt Management for LLM-Driven Software Engineeringcited by 1
  7. 2025LSPRAG: LSP-Guided RAG for Language-Agnostic Real-Time Unit Test Generationcited by 1
  8. 2026SCATE: Learning to Supervise Coding Agents for Cost-Effective Test Generationcited by 1
  9. 2025LibRec: Benchmarking Retrieval-Augmented LLMs for Library Migration Recommendationscited by 0
  10. 2025Assertion-Aware Test Code Summarization with Large Language Modelscited by 0
  11. 2026Library-Aware Doubles and Iterative Repair for Large Language Model-Generated Unit Tests in OpenSIL Firmwarecited by 0