# To What Extent do Deep Learning-based Code Recommenders Generate Predictions by Cloning Code from the Training Set?

arXiv: https://arxiv.org/abs/2204.06894 · HTML: https://mlsanity.com/paper/2204.06894/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2022.

## Built on by (most cited first, 11 of 11)

- [Automatic Generation of Programming Exercises and Code Explanations using Large Language Models](https://mlsanity.com/paper/2206.11861/) (2022)
- [MELTing point: Mobile Evaluation of Language Transformers](https://mlsanity.com/paper/2403.12844/) (2024)
- [Exploring the Robustness of Large Language Models for Solving Programming Problems](https://mlsanity.com/paper/2306.14583/) (2023)
- [From Copilot to Pilot: Towards AI Supported Software Development](https://mlsanity.com/paper/2303.04142/) (2023)
- [Do Large Language Models Pay Similar Attention Like Human Programmers When Generating Code?](https://mlsanity.com/paper/2306.01220/) (2023)
- [All You Need Is Logs: Improving Code Completion by Learning from Anonymous IDE Usage Logs](https://mlsanity.com/paper/2205.10692/) (2022)
- [Trained Without My Consent: Detecting Code Inclusion In Language Models Trained on Code](https://mlsanity.com/paper/2402.09299/) (2024)
- [GitHub Copilot: the perfect Code compLeeter?](https://mlsanity.com/paper/2406.11326/) (2024)
- [Promise and Peril of Collaborative Code Generation Models: Balancing Effectiveness and Memorization](https://mlsanity.com/paper/2409.12020/) (2024)
- [Are AI Coders Snitches? An Empirical Study of Pretraining Data Detection on Code Large Language Models](https://mlsanity.com/paper/2507.17389/) (2025)
- [Understanding Secret Leakage Risks in Code LLMs: A Tokenization Perspective](https://mlsanity.com/paper/2604.17814/) (2026)

## Abstract (arXiv)

Deep Learning (DL) models have been widely used to support code completion. These models, once properly trained, can take as input an incomplete code component (e.g., an incomplete function) and predict the missing tokens to finalize it. GitHub Copilot is an example of code recommender built by training a DL model on millions of open source repositories: The source code of these repositories acts as training data, allowing the model to learn "how to program". The usage of such a code is usually regulated by Free and Open Source Software (FOSS) licenses, that establish under which conditions the licensed code can be redistributed or modified. As of Today, it is unclear whether the code generated by DL models trained on open source code should be considered as "new" or as "derivative" work, with possible implications on license infringements. In this work, we run a large-scale study investigating the extent to which DL models tend to clone code from their training set when recommending code completions. Such an exploratory study can help in assessing the magnitude of the potential licensing issues mentioned before: If these models tend to generate new code that is unseen in the training set, then licensing issues are unlikely to occur. Otherwise, a revision of these licenses urges to regulate how the code generated by these models should be treated when used, for example, in a commercial setting. Highlights from our results show that ~10% to ~0.1% of the predictions generated by a state-of-the-art DL-based code completion tool are Type-1 clones of instances in the training set, depending on the size of the predicted code. Long predictions are unlikely to be cloned.