# CodeMark: Imperceptible Watermarking for Code Datasets against Neural Code Completion Models

arXiv: https://arxiv.org/abs/2308.14401 · HTML: https://mlsanity.com/paper/2308.14401/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security.
- Builds on 27 of its 54 references that are in the map.
- First posted on arXiv 28 Aug 2023.

## Placed in

- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks](https://mlsanity.com/paper/1804.00792/) (2018)
- [Spectral Signatures in Backdoor Attacks](https://mlsanity.com/paper/1811.00636/) (2018)
- [Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering](https://mlsanity.com/paper/1811.03728/) (2018)
- [The Stack: 3 TB of permissively licensed source code](https://mlsanity.com/paper/2211.15533/) (2022)
- [Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring](https://mlsanity.com/paper/1802.04633/) (2018)
- [Clean-Label Backdoor Attacks on Video Recognition Models](https://mlsanity.com/paper/2003.03030/) (2020)
- [You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion](https://mlsanity.com/paper/2007.02220/) (2020)
- [Protecting Intellectual Property of Language Generation APIs with Lexical Watermark](https://mlsanity.com/paper/2112.02701/) (2021)
- [Natural Attack for Pre-trained Models of Code](https://mlsanity.com/paper/2201.08698/) (2022)
- [Adversarial Examples for Models of Code](https://mlsanity.com/paper/1910.07517/) (2019)
- [Self-Supervised Contrastive Learning for Code Retrieval and Summarization via Semantic-Preserving Transformations](https://mlsanity.com/paper/2009.02731/) (2020)
- [CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning](https://mlsanity.com/paper/2110.12925/) (2021)
- [On the Generalizability of Neural Program Models with respect to Semantic-Preserving Program Transformations](https://mlsanity.com/paper/2008.01566/) (2020)
- [Backdoors in Neural Models of Source Code](https://mlsanity.com/paper/2006.06841/) (2020)
- [A Survey on Deep Learning for Software Engineering](https://mlsanity.com/paper/2011.14597/) (2020)
- [Open-sourced Dataset Protection via Backdoor Watermarking](https://mlsanity.com/paper/2010.05821/) (2020)
- [Challenging Machine Learning-based Clone Detectors via Semantic-preserving Code Transformations](https://mlsanity.com/paper/2111.10793/) (2021)
- [On the Effectiveness of Dataset Watermarking in Adversarial Settings](https://mlsanity.com/paper/2202.12506/) (2022)
- [Public Git Archive: a Big Code dataset for all](https://mlsanity.com/paper/1803.10144/) (2018)
- [Robust Black-box Watermarking for Deep NeuralNetwork using Inverse Document Frequency](https://mlsanity.com/paper/2103.05590/) (2021)
- [STRATA: Simple, Gradient-Free Attacks for Models of Code](https://mlsanity.com/paper/2009.13562/) (2020)

## Built on by (most cited first, 30 of 34)

- [A Survey of Text Watermarking in the Era of Large Language Models](https://mlsanity.com/paper/2312.07913/) (2023)
- [Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2502.06872/) (2025)
- [Between Lines of Code: Unraveling the Distinct Patterns of Machine and Human Programmers](https://mlsanity.com/paper/2401.06461/) (2024)
- [CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Code](https://mlsanity.com/paper/2404.15639/) (2024)
- [Double-I Watermark: Protecting Model Copyright for LLM Fine-tuning](https://mlsanity.com/paper/2402.14883/) (2024)
- [Does Your Neural Code Completion Model Use My Code? A Membership Inference Approach](https://mlsanity.com/paper/2404.14296/) (2024)
- [Marking Code Without Breaking It: Code Watermarking for Detecting LLM-Generated Code](https://mlsanity.com/paper/2502.18851/) (2025)
- [Pitfalls in Language Models for Code Intelligence: A Taxonomy and Survey](https://mlsanity.com/paper/2310.17903/) (2023)
- [Watermark under Fire: A Robustness Evaluation of LLM Watermarking](https://mlsanity.com/paper/2411.13425/) (2024)
- [Inevitable Trade-off between Watermark Strength and Speculative Sampling Efficiency for Language Models](https://mlsanity.com/paper/2410.20418/) (2024)
- [Efficient and Universal Watermarking for LLM-Generated Code Detection](https://mlsanity.com/paper/2402.07518/) (2024)
- [Invisible Entropy: Towards Safe and Efficient Low-Entropy LLM Watermarking](https://mlsanity.com/paper/2505.14112/) (2025)
- [WaterMod: Modular Token-Rank Partitioning for Probability-Balanced LLM Watermarking](https://mlsanity.com/paper/2511.07863/) (2025)
- [Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigm](https://mlsanity.com/paper/2509.07287/) (2025)
- [Sequential Behavioral Watermarking for LLM Agents](https://mlsanity.com/paper/2605.11036/) (2026)
- [Lost in Overlap: Exploring Logit-based Watermark Collision in LLMs](https://mlsanity.com/paper/2403.10020/) (2024)
- [Watermarking LLM Agent Trajectories](https://mlsanity.com/paper/2602.18700/) (2026)
- [From Intentions to Techniques: A Comprehensive Taxonomy and Challenges in Text Watermarking for Large Language Models](https://mlsanity.com/paper/2406.11106/) (2024)
- [CodeCipher: Learning to Obfuscate Source Code Against LLMs](https://mlsanity.com/paper/2410.05797/) (2024)
- [When Only the Final Text Survives: Implicit Execution Tracing for Multi-Agent Auditing](https://mlsanity.com/paper/2603.17445/) (2026)
- [Beyond Dataset Watermarking: Model-Level Copyright Protection for Code Summarization Models](https://mlsanity.com/paper/2410.14102/) (2024)
- [Watermarking Large Language Models and the Generated Content: Opportunities and Challenges](https://mlsanity.com/paper/2410.19096/) (2024)
- [SoK: Are Watermarks in LLMs Ready for Deployment?](https://mlsanity.com/paper/2506.05594/) (2025)
- [SSG: Logit-Balanced Vocabulary Partitioning for LLM Watermarking](https://mlsanity.com/paper/2604.22438/) (2026)
- [MCGMark: An Encodable and Robust Online Watermark for Tracing LLM-Generated Malicious Code](https://mlsanity.com/paper/2408.01354/) (2024)
- [Effective Code Membership Inference for Code Completion Models via Adversarial Prompts](https://mlsanity.com/paper/2511.15107/) (2025)
- [Robust and Secure Code Watermarking for Large Language Models via ML/Crypto Codesign](https://mlsanity.com/paper/2502.02068/) (2025)
- [Towards Generalized and Stealthy Watermarking for Generative Code Models](https://mlsanity.com/paper/2506.20926/) (2025)
- [Disappearing Ink: Obfuscation Breaks N-gram Code Watermarks in Theory and Practice](https://mlsanity.com/paper/2507.05512/) (2025)
- [CODE ACROSTIC: Robust Watermarking for Code Generation](https://mlsanity.com/paper/2512.14753/) (2025)

## Abstract (arXiv)

Code datasets are of immense value for training neural-network-based code completion models, where companies or organizations have made substantial investments to establish and process these datasets. Unluckily, these datasets, either built for proprietary or public usage, face the high risk of unauthorized exploits, resulting from data leakages, license violations, etc. Even worse, the ``black-box'' nature of neural models sets a high barrier for externals to audit their training datasets, which further connives these unauthorized usages. Currently, watermarking methods have been proposed to prohibit inappropriate usage of image and natural language datasets. However, due to domain specificity, they are not directly applicable to code datasets, leaving the copyright protection of this emerging and important field of code data still exposed to threats. To fill this gap, we propose a method, named CodeMark, to embed user-defined imperceptible watermarks into code datasets to trace their usage in training neural code completion models. CodeMark is based on adaptive semantic-preserving transformations, which preserve the exact functionality of the code data and keep the changes covert against rule-breakers. We implement CodeMark in a toolkit and conduct an extensive evaluation of code completion models. CodeMark is validated to fulfill all desired properties of practical watermarks, including harmlessness to model accuracy, verifiability, robustness, and imperceptibility.