# CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning

arXiv: https://arxiv.org/abs/2110.12925 · HTML: https://mlsanity.com/paper/2110.12925/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security.
- Builds on 26 of its 46 references that are in the map.
- First posted on arXiv 25 Oct 2021.

## Placed in

- LLM safety and security / [Backdoor and data poisoning attacks](https://mlsanity.com/safety/backdoor-and-data-poisoning-attacks/)

## Builds on (most cited first, 26 of 26)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks](https://mlsanity.com/paper/1804.00792/) (2018)
- [Spectral Signatures in Backdoor Attacks](https://mlsanity.com/paper/1811.00636/) (2018)
- [Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering](https://mlsanity.com/paper/1811.03728/) (2018)
- [Certified Defenses for Data Poisoning Attacks](https://mlsanity.com/paper/1706.03691/) (2017)
- [Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger](https://mlsanity.com/paper/2105.12400/) (2021)
- [Clean-Label Backdoor Attacks on Video Recognition Models](https://mlsanity.com/paper/2003.03030/) (2020)
- [Stronger Data Poisoning Attacks Break Data Sanitization Defenses](https://mlsanity.com/paper/1811.00741/) (2018)
- [A Transformer-based Approach for Source Code Summarization](https://mlsanity.com/paper/2005.00653/) (2020)
- [You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion](https://mlsanity.com/paper/2007.02220/) (2020)
- [Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?](https://mlsanity.com/paper/1904.05506/) (2019)
- [Radioactive data: tracing through training](https://mlsanity.com/paper/2002.00937/) (2020)
- [TreeGen: A Tree-Based Transformer Architecture for Code Generation](https://mlsanity.com/paper/1911.09983/) (2019)
- [Backdoors in Neural Models of Source Code](https://mlsanity.com/paper/2006.06841/) (2020)
- [A Survey on Deep Learning for Software Engineering](https://mlsanity.com/paper/2011.14597/) (2020)
- [Preventing Unauthorized Use of Proprietary Data: Poisoning for Secure Dataset Release](https://mlsanity.com/paper/2103.02683/) (2021)
- [Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder](https://mlsanity.com/paper/2010.02684/) (2020)
- [Open-sourced Dataset Protection via Backdoor Watermarking](https://mlsanity.com/paper/2010.05821/) (2020)
- [Copyright in Generative Deep Learning](https://mlsanity.com/paper/2105.09266/) (2021)
- [A Study of Potential Code Borrowing and License Violations in Java Projects on GitHub](https://mlsanity.com/paper/2002.05237/) (2020)
- [STRATA: Simple, Gradient-Free Attacks for Models of Code](https://mlsanity.com/paper/2009.13562/) (2020)
- [Muzeel: A Dynamic JavaScript Analyzer for Dead Code Elimination in Today's Web](https://mlsanity.com/paper/2106.08948/) (2021)

## Built on by (most cited first, 30 of 47)

- [Large Language Models for Code: Security Hardening and Adversarial Testing](https://mlsanity.com/paper/2302.05319/) (2023)
- [A Survey of Text Watermarking in the Era of Large Language Models](https://mlsanity.com/paper/2312.07913/) (2023)
- [BadEdit: Backdooring large language models by model editing](https://mlsanity.com/paper/2403.13355/) (2024)
- [Stealthy Backdoor Attack for Code Models](https://mlsanity.com/paper/2301.02496/) (2023)
- [CodeMark: Imperceptible Watermarking for Code Datasets against Neural Code Completion Models](https://mlsanity.com/paper/2308.14401/) (2023)
- [Vulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attacks](https://mlsanity.com/paper/2308.04451/) (2023)
- [Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2502.06872/) (2025)
- [Stable Unlearnable Example: Enhancing the Robustness of Unlearnable Examples via Stable Error-Minimizing Noise](https://mlsanity.com/paper/2311.13091/) (2023)
- [Multi-target Backdoor Attacks for Code Pre-trained Models](https://mlsanity.com/paper/2306.08350/) (2023)
- [Backdooring Neural Code Search](https://mlsanity.com/paper/2305.17506/) (2023)
- [Don't Complete It! Preventing Unhelpful Code Completion for Productive and Sustainable Neural Code Completion Systems](https://mlsanity.com/paper/2209.05948/) (2022)
- [SALAD: Systematic Assessment of Machine Unlearning on LLM-Aided Hardware Design](https://mlsanity.com/paper/2506.02089/) (2025)
- [Data Isotopes for Data Provenance in DNNs](https://mlsanity.com/paper/2208.13893/) (2022)
- [Does Your Neural Code Completion Model Use My Code? A Membership Inference Approach](https://mlsanity.com/paper/2404.14296/) (2024)
- [VeriLeaky: Navigating IP Protection vs Utility in Fine-Tuning for LLM-Driven Verilog Coding](https://mlsanity.com/paper/2503.13116/) (2025)
- [The (ab)use of Open Source Code to Train Large Language Models](https://mlsanity.com/paper/2302.13681/) (2023)
- [Pitfalls in Language Models for Code Intelligence: A Taxonomy and Survey](https://mlsanity.com/paper/2310.17903/) (2023)
- [Watermark under Fire: A Robustness Evaluation of LLM Watermarking](https://mlsanity.com/paper/2411.13425/) (2024)
- [Medical Unlearnable Examples: Securing Medical Data from Unauthorized Training via Sparsity-Aware Local Masking](https://mlsanity.com/paper/2403.10573/) (2024)
- [Inevitable Trade-off between Watermark Strength and Speculative Sampling Efficiency for Language Models](https://mlsanity.com/paper/2410.20418/) (2024)
- [Privacy and Copyright Protection in Generative AI: A Lifecycle Perspective](https://mlsanity.com/paper/2311.18252/) (2023)
- [Invisible Entropy: Towards Safe and Efficient Low-Entropy LLM Watermarking](https://mlsanity.com/paper/2505.14112/) (2025)
- [WaterMod: Modular Token-Rank Partitioning for Probability-Balanced LLM Watermarking](https://mlsanity.com/paper/2511.07863/) (2025)
- [Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigm](https://mlsanity.com/paper/2509.07287/) (2025)
- [Sequential Behavioral Watermarking for LLM Agents](https://mlsanity.com/paper/2605.11036/) (2026)
- [Lost in Overlap: Exploring Logit-based Watermark Collision in LLMs](https://mlsanity.com/paper/2403.10020/) (2024)
- [Measuring Impacts of Poisoning on Model Parameters and Embeddings for Large Language Models of Code](https://mlsanity.com/paper/2405.11466/) (2024)
- [Promise and Peril of Collaborative Code Generation Models: Balancing Effectiveness and Memorization](https://mlsanity.com/paper/2409.12020/) (2024)
- [CredID: Credible Multi-Bit Watermark for Large Language Models Identification](https://mlsanity.com/paper/2412.03107/) (2024)
- [Watermarking LLM Agent Trajectories](https://mlsanity.com/paper/2602.18700/) (2026)

## Abstract (arXiv)

Github Copilot, trained on billions of lines of public code, has recently become the buzzword in the computer science research and practice community. Although it is designed to help developers implement safe and effective code with powerful intelligence, practitioners and researchers raise concerns about its ethical and security problems, e.g., should the copyleft licensed code be freely leveraged or insecure code be considered for training in the first place? These problems pose a significant impact on Copilot and other similar products that aim to learn knowledge from large-scale open-source code through deep learning models, which are inevitably on the rise with the fast development of artificial intelligence. To mitigate such impacts, we argue that there is a need to invent effective mechanisms for protecting open-source code from being exploited by deep learning models. Here, we design and implement a prototype, CoProtector, which utilizes data poisoning techniques to arm source code repositories for defending against such exploits. Our large-scale experiments empirically show that CoProtector is effective in achieving its purpose, significantly reducing the performance of Copilot-like deep learning models while being able to stably reveal the secretly embedded watermark backdoors.