mlsanityData to 7 Oct 2026

Harnessing the Power of LLMs: Automating Unit Test Generation for High-Performance Computing

arXiv 2407.05202 ↗First posted 6 Jul 2024cs.SE cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Unit testing is crucial in software engineering for ensuring quality. However, it's not widely used in parallel and high-performance computing software, particularly scientific applications, due to their smaller, diverse user base and complex logic. These factors make unit testing challenging and expensive, as it requires specialized knowledge and existing automated tools are often ineffective. To address this, we propose an automated method for generating unit tests for such software, considering their unique features like complex logic and parallel processing. Recently, large language models (LLMs) have shown promise in coding and testing. We explored the capabilities of Davinci (text-davinci-002) and ChatGPT (gpt-3.5-turbo) in creating unit tests for C++ parallel programs. Our results show that LLMs can generate mostly correct and comprehensive unit tests, although they have some limitations, such as repetitive assertions and blank test cases.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 22 of its 63 references that are in the map. First posted on arXiv 6 Jul 2024.

Builds on

22 of 63 references.

References · 22

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 417
  3. 2022CodeT: Code Generation with Generated Testscited by 369
  4. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  5. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 327
  6. 2023Self-collaboration Code Generation via ChatGPTcited by 255
  7. 2020IntelliCode Compose: Code Generation Using Transformercited by 225
  8. 2023An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generationcited by 193
  9. 2022Do Users Write More Insecure Code with AI Assistants?cited by 186
  10. 2020Unit Test Case Generation with Transformers and Focal Contextcited by 118
  11. 2023No More Manual Tests? Evaluating and Improving ChatGPT for Unit Test Generationcited by 82
  12. 2023Can ChatGPT replace StackOverflow? A Study on Robustness and Reliability of Large Language Model Code Generationcited by 72
10 more
  1. 2021Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Taskscited by 70
  2. 2018Fast, Diverse and Accurate Image Captioning Guided By Part-of-Speechcited by 55
  3. 2022Interactive Code Generation via Test-Driven User-Intent Formalizationcited by 50
  4. 2023Large Language Models and Simple, Stupid Bugscited by 45
  5. 2022Code Generation Tools (Almost) for Free? A Study of Few-Shot, Pre-Trained Language Models on Codecited by 39
  6. 2017Diverse and Accurate Image Description Using a Variational Auto-Encoder with an Additive Gaussian Encoding Spacecited by 33
  7. 2023Evaluation of OpenAI Codex for HPC Parallel Programming Models Kernel Generationcited by 22
  8. 2023The potential of LLMs for coding with low-resource and domain-specific programming languagescited by 11
  9. 2019On the Relation Between Unit Testing and Code Qualitycited by 3
  10. 2021(R)SE challenges in HPCcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024LogBabylon: A Unified Framework for Cross-Log File Integration and Analysiscited by 1
  2. 2025HPCAgentTester: A Multi-Agent LLM Approach for Enhanced HPC Unit Test Generationcited by 0
  3. 2026Library-Aware Doubles and Iterative Repair for Large Language Model-Generated Unit Tests in OpenSIL Firmwarecited by 0