# Investigating the Efficacy of Large Language Models for Code Clone Detection

arXiv: https://arxiv.org/abs/2401.13802 · HTML: https://mlsanity.com/paper/2401.13802/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 27 references that are in the map.
- First posted on arXiv 24 Jan 2024.

## Builds on (most cited first, 11 of 11)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021)
- [GraphCodeBERT: Pre-training Code Representations with Data Flow](https://mlsanity.com/paper/2009.08366/) (2020)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [A Systematic Evaluation of Large Language Models of Code](https://mlsanity.com/paper/2202.13169/) (2022)
- [SynCoBERT: Syntax-Guided Multi-Modal Contrastive Pre-Training for Code Representation](https://mlsanity.com/paper/2108.04556/) (2021)
- [Code Generation Tools (Almost) for Free? A Study of Few-Shot, Pre-Trained Language Models on Code](https://mlsanity.com/paper/2206.01335/) (2022)
- [Towards Understanding the Capability of Large Language Models on Code Clone Detection: A Survey](https://mlsanity.com/paper/2308.01191/) (2023)
- [Generalizability of Code Clone Detection on CodeBERT](https://mlsanity.com/paper/2208.12588/) (2022)
- [Unveiling the potential of large language models in generating semantic and cross-language clones](https://mlsanity.com/paper/2309.06424/) (2023)

## Built on by (most cited first, 7 of 7)

- [The Struggles of LLMs in Cross-lingual Code Clone Detection](https://mlsanity.com/paper/2408.04430/) (2024)
- [GenCode: A Generic Data Augmentation Framework for Boosting Deep Learning-Based Code Understanding](https://mlsanity.com/paper/2402.15769/) (2024)
- [Should Code Models Learn Pedagogically? A Preliminary Evaluation of Curriculum Learning for Real-World Software Engineering Tasks](https://mlsanity.com/paper/2502.03806/) (2025)
- [LSem2Vec: A Simple yet Effective Two-Stage Approach for Source Code Embedding](https://mlsanity.com/paper/2409.14644/) (2024)
- [A Preliminary Study of Multilingual Code Language Models for Code Generation Task Using Translated Benchmarks](https://mlsanity.com/paper/2411.15470/) (2024)
- [Resource-Efficient Automatic Software Vulnerability Assessment via Knowledge Distillation and Particle Swarm Optimization](https://mlsanity.com/paper/2508.02840/) (2025)
- [Selecting and Combining Large Language Models for Scalable Code Clone Detection](https://mlsanity.com/paper/2510.15480/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable success in various natural language processing and software engineering tasks, such as code generation. The LLMs are mainly utilized in the prompt-based zero/few-shot paradigm to guide the model in accomplishing the task. GPT-based models are one of the popular ones studied for tasks such as code comment generation or test generation. These tasks are `generative' tasks. However, there is limited research on the usage of LLMs for `non-generative' tasks such as classification using the prompt-based paradigm. In this preliminary exploratory study, we investigated the applicability of LLMs for Code Clone Detection (CCD), a non-generative task. By building a mono-lingual and cross-lingual CCD dataset derived from CodeNet, we first investigated two different prompts using ChatGPT to detect Type-4 code clones in Java-Java and Java-Ruby pairs in a zero-shot setting. We then conducted an analysis to understand the strengths and weaknesses of ChatGPT in CCD. ChatGPT surpasses the baselines in cross-language CCD attaining an F1-score of 0.877 and achieves comparable performance to fully fine-tuned models for mono-lingual CCD, with an F1-score of 0.878. Also, the prompt and the difficulty level of the problems has an impact on the performance of ChatGPT. Finally we provide insights and future directions based on our initial analysis