# The (ab)use of Open Source Code to Train Large Language Models

arXiv: https://arxiv.org/abs/2302.13681 · HTML: https://mlsanity.com/paper/2302.13681/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 5 references that are in the map.
- First posted on arXiv 27 Feb 2023.

## Builds on (most cited first, 5 of 5)

- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequences](https://mlsanity.com/paper/2202.06689/) (2022)
- [CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning](https://mlsanity.com/paper/2110.12925/) (2021)
- [Extending Source Code Pre-Trained Language Models to Summarise Decompiled Binaries](https://mlsanity.com/paper/2301.01701/) (2023)

## Built on by (most cited first, 10 of 10)

- [Traces of Memorisation in Large Language Models for Code](https://mlsanity.com/paper/2312.11658/) (2023)
- [Mapping the Ethics of Generative AI: A Comprehensive Scoping Review](https://mlsanity.com/paper/2402.08323/) (2024)
- [LiCoEval: Evaluating LLMs on License Compliance in Code Generation](https://mlsanity.com/paper/2408.02487/) (2024)
- [An Exploratory Investigation into Code License Infringements in Large Language Model Training Datasets](https://mlsanity.com/paper/2403.15230/) (2024)
- [Code Red! On the Harmfulness of Applying Off-the-shelf Large Language Models to Programming Tasks](https://mlsanity.com/paper/2504.01850/) (2025)
- [How Much Do Code Language Models Remember? An Investigation on Data Extraction Attacks before and after Fine-tuning](https://mlsanity.com/paper/2501.17501/) (2025)
- [Mitigating Sensitive Information Leakage in LLMs4Code through Machine Unlearning](https://mlsanity.com/paper/2502.05739/) (2025)
- [Model See, Model Do? Exposure-Aware Evaluation of Bug-vs-Fix Preference in Code LLMs](https://mlsanity.com/paper/2601.10496/) (2026)
- [AST-PAC: AST-guided Membership Inference for Code](https://mlsanity.com/paper/2602.13240/) (2026)
- [Understanding Secret Leakage Risks in Code LLMs: A Tokenization Perspective](https://mlsanity.com/paper/2604.17814/) (2026)

## Abstract (arXiv)

In recent years, Large Language Models (LLMs) have gained significant popularity due to their ability to generate human-like text and their potential applications in various fields, such as Software Engineering. LLMs for Code are commonly trained on large unsanitized corpora of source code scraped from the Internet. The content of these datasets is memorized and emitted by the models, often in a verbatim manner. In this work, we will discuss the security, privacy, and licensing implications of memorization. We argue why the use of copyleft code to train LLMs is a legal and ethical dilemma. Finally, we provide four actionable recommendations to address this issue.