# Assessing the Code Clone Detection Capability of Large Language Models

arXiv: https://arxiv.org/abs/2407.02402 · HTML: https://mlsanity.com/paper/2407.02402/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 42 references that are in the map.
- First posted on arXiv 2 Jul 2024.

## Builds on (most cited first, 11 of 11)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Uncovering ChatGPT's Capabilities in Recommender Systems](https://mlsanity.com/paper/2305.02182/) (2023)
- [Explainable Automated Debugging via Large Language Model-driven Scientific Debugging](https://mlsanity.com/paper/2304.02195/) (2023)
- [Translating Radiology Reports into Plain Language using ChatGPT and GPT-4 with Prompt Learning: Promising Results, Limitations, and Potential](https://mlsanity.com/paper/2303.09038/) (2023)
- [Automatic Code Documentation Generation Using GPT-3](https://mlsanity.com/paper/2209.02235/) (2022)
- [Oreo: Detection of Clones in the Twilight Zone](https://mlsanity.com/paper/1806.05837/) (2018)
- [Zero Shot Learning for Code Education: Rubric Sampling with Deep Learning Inference](https://mlsanity.com/paper/1809.01357/) (2018)
- [Comparing Software Developers with ChatGPT: An Empirical Investigation](https://mlsanity.com/paper/2305.11837/) (2023)
- [Linguistically Informed ChatGPT Prompts to Enhance Japanese-Chinese Machine Translation: A Case Study on Attributive Clauses](https://mlsanity.com/paper/2303.15587/) (2023)

## Built on by (most cited first, 1 of 1)

- [Type-IV Code Clone Detection via Layer-Wise Non-Contrastive Representation Learning](https://mlsanity.com/paper/2609.17338/) (2026)

## Abstract (arXiv)

This study aims to assess the performance of two advanced Large Language Models (LLMs), GPT-3.5 and GPT-4, in the task of code clone detection. The evaluation involves testing the models on a variety of code pairs of different clone types and levels of similarity, sourced from two datasets: BigCloneBench (human-made) and GPTCloneBench (LLM-generated). Findings from the study indicate that GPT-4 consistently surpasses GPT-3.5 across all clone types. A correlation was observed between the GPTs' accuracy at identifying code clones and code similarity, with both GPT models exhibiting low effectiveness in detecting the most complex Type-4 code clones. Additionally, GPT models demonstrate a higher performance identifying code clones in LLM-generated code compared to humans-generated code. However, they do not reach impressive accuracy. These results emphasize the imperative for ongoing enhancements in LLM capabilities, particularly in the recognition of code clones and in mitigating their predisposition towards self-generated code clones--which is likely to become an issue as software engineers are more numerous to leverage LLM-enabled code generation and code refactoring tools.