# Breaking the Silence: the Threats of Using LLMs in Software Engineering

arXiv: https://arxiv.org/abs/2312.08055 · HTML: https://mlsanity.com/paper/2312.08055/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 46 references that are in the map.
- First posted on arXiv 13 Dec 2023.

## Builds on (most cited first, 21 of 21)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023)
- [Multi-lingual Evaluation of Code Generation Models](https://mlsanity.com/paper/2210.14868/) (2022)
- [Keep the Conversation Going: Fixing 162 out of 337 bugs for 0.42 each using ChatGPT](https://mlsanity.com/paper/2304.00385/) (2023)
- [Unit Test Case Generation with Transformers and Focal Context](https://mlsanity.com/paper/2009.05617/) (2020)
- [A Systematic Review of Green AI](https://mlsanity.com/paper/2301.11047/) (2023)
- [Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?](https://mlsanity.com/paper/2204.04741/) (2022)
- [Using Large Language Models to Generate JUnit Tests: An Empirical Study](https://mlsanity.com/paper/2305.00418/) (2023)
- [Large Language Models and Simple, Stupid Bugs](https://mlsanity.com/paper/2303.11455/) (2023)
- [On the Challenges of Using Black-Box APIs for Toxicity Evaluation in Research](https://mlsanity.com/paper/2304.12397/) (2023)
- [A3Test: Assertion-Augmented Automated Test Case Generation](https://mlsanity.com/paper/2302.10352/) (2023)
- [ChatGPT vs SBST: A Comparative Assessment of Unit Test Suite Generation](https://mlsanity.com/paper/2307.00588/) (2023)
- [Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibility](https://mlsanity.com/paper/2305.10235/) (2023)
- [Large Language Models in Fault Localisation](https://mlsanity.com/paper/2308.15276/) (2023)
- [Codex Hacks HackerRank: Memorization Issues and a Framework for Code Synthesis Evaluation](https://mlsanity.com/paper/2212.02684/) (2022)
- [Training Data Leakage Analysis in Language Models](https://mlsanity.com/paper/2101.05405/) (2021)
- [JUGE: An Infrastructure for Benchmarking Java Unit Test Generators](https://mlsanity.com/paper/2106.07520/) (2021)

## Abstract (arXiv)

Large Language Models (LLMs) have gained considerable traction within the Software Engineering (SE) community, impacting various SE tasks from code completion to test generation, from program repair to code summarization. Despite their promise, researchers must still be careful as numerous intricate factors can influence the outcomes of experiments involving LLMs. This paper initiates an open discussion on potential threats to the validity of LLM-based research including issues such as closed-source models, possible data leakage between LLM training data and research evaluation, and the reproducibility of LLM-based findings. In response, this paper proposes a set of guidelines tailored for SE researchers and Language Model (LM) providers to mitigate these concerns. The implications of the guidelines are illustrated using existing good practices followed by LLM providers and a practical example for SE researchers in the context of test case generation.