mlsanityData to 7 Oct 2026

CodeMark: Imperceptible Watermarking for Code Datasets against Neural Code Completion Models

arXiv 2308.14401 ↗First posted 28 Aug 2023cs.SE cs.AIcited by 28

On the map: LLM safety and security → Watermarking generated content and models

Abstract (arXiv)

Code datasets are of immense value for training neural-network-based code completion models, where companies or organizations have made substantial investments to establish and process these datasets. Unluckily, these datasets, either built for proprietary or public usage, face the high risk of unauthorized exploits, resulting from data leakages, license violations, etc. Even worse, the ``black-box'' nature of neural models sets a high barrier for externals to audit their training datasets, which further connives these unauthorized usages. Currently, watermarking methods have been proposed to prohibit inappropriate usage of image and natural language datasets. However, due to domain specificity, they are not directly applicable to code datasets, leaving the copyright protection of this emerging and important field of code data still exposed to threats. To fill this gap, we propose a method, named CodeMark, to embed user-defined imperceptible watermarks into code datasets to trace their usage in training neural code completion models. CodeMark is based on adaptive semantic-preserving transformations, which preserve the exact functionality of the code data and keep the changes covert against rule-breakers. We implement CodeMark in a toolkit and conduct an extensive evaluation of code completion models. CodeMark is validated to fulfill all desired properties of practical watermarks, including harmlessness to model accuracy, verifiability, robustness, and imperceptibility.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. Builds on 27 of its 54 references that are in the map. First posted on arXiv 28 Aug 2023.

Builds on

27 of 54 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Protecting Intellectual Property of Language Generation APIs with Lexical Watermark5 Dec

Other references · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  3. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  4. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  5. 2023StarCoder: may the source be with you!cited by 790
  6. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  7. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  8. 2018Spectral Signatures in Backdoor Attackscited by 457
  9. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  10. 2022The Stack: 3 TB of permissively licensed source codecited by 354
  11. 2018Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooringcited by 334
  12. 2020Clean-Label Backdoor Attacks on Video Recognition Modelscited by 162
14 more
  1. 2020You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completioncited by 112
  2. 2022Natural Attack for Pre-trained Models of Codecited by 72
  3. 2019Adversarial Examples for Models of Codecited by 61
  4. 2020Self-Supervised Contrastive Learning for Code Retrieval and Summarization via Semantic-Preserving Transformationscited by 49
  5. 2020On the Generalizability of Neural Program Models with respect to Semantic-Preserving Program Transformationscited by 41
  6. 2021CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoningcited by 39
  7. 2020A Survey on Deep Learning for Software Engineeringcited by 37
  8. 2020Open-sourced Dataset Protection via Backdoor Watermarkingcited by 30
  9. 2020Backdoors in Neural Models of Source Codecited by 27
  10. 2021Challenging Machine Learning-based Clone Detectors via Semantic-preserving Code Transformationscited by 15
  11. 2022On the Effectiveness of Dataset Watermarking in Adversarial Settingscited by 11
  12. 2021Robust Black-box Watermarking for Deep NeuralNetwork using Inverse Document Frequencycited by 6
  13. 2018Public Git Archive: a Big Code dataset for allcited by 5
  14. 2020STRATA: Simple, Gradient-Free Attacks for Models of Codecited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 10; 2025: 10; 2026: 6.

28 papers.

Sort:

By area:

  1. 2023A Survey of Text Watermarking in the Era of Large Language Modelscited by 102
  2. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Surveycited by 29
  3. 2024Between Lines of Code: Unraveling the Distinct Patterns of Machine and Human Programmerscited by 27
  4. 2024CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Codecited by 21
  5. 2024Double-I Watermark: Protecting Model Copyright for LLM Fine-tuningcited by 18
  6. 2023Pitfalls in Language Models for Code Intelligence: A Taxonomy and Surveycited by 9
  7. 2024Does Your Neural Code Completion Model Use My Code? A Membership Inference Approachcited by 9
  8. 2024Inevitable Trade-off between Watermark Strength and Speculative Sampling Efficiency for Language Modelscited by 7
  9. 2024Watermark under Fire: A Robustness Evaluation of LLM Watermarkingcited by 7
  10. 2025Marking Code Without Breaking It: Code Watermarking for Detecting LLM-Generated Codecited by 7
  11. 2025Invisible Entropy: Towards Safe and Efficient Low-Entropy LLM Watermarkingcited by 5
  12. 2025Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigmcited by 4
16 more
  1. 2025WaterMod: Modular Token-Rank Partitioning for Probability-Balanced LLM Watermarkingcited by 4
  2. 2024From Intentions to Techniques: A Comprehensive Taxonomy and Challenges in Text Watermarking for Large Language Modelscited by 3
  3. 2024CodeCipher: Learning to Obfuscate Source Code Against LLMscited by 3
  4. 2026Watermarking LLM Agent Trajectoriescited by 3
  5. 2026Sequential Behavioral Watermarking for LLM Agentscited by 3
  6. 2024Lost in Overlap: Exploring Logit-based Watermark Collision in LLMscited by 2
  7. 2024Watermarking Large Language Models and the Generated Content: Opportunities and Challengescited by 2
  8. 2025SoK: Are Watermarks in LLMs Ready for Deployment?cited by 2
  9. 2026When Only the Final Text Survives: Implicit Execution Tracing for Multi-Agent Auditingcited by 2
  10. 2025Robust and Secure Code Watermarking for Large Language Models via ML/Crypto Codesigncited by 0
  11. 2025Disappearing Ink: Obfuscation Breaks N-gram Code Watermarks in Theory and Practicecited by 0
  12. 2025Effective Code Membership Inference for Code Completion Models via Adversarial Promptscited by 0
  13. 2025CODE ACROSTIC: Robust Watermarking for Code Generationcited by 0
  14. 2026Semantic Differentiation for Tackling Challenges in Watermarking Low-Entropy Constrained Generation Outputscited by 0
  15. 2026Optimal Multi-bit Generative Watermarking Schemes Under Worst-Case False-Alarm Constraintscited by 0
  16. 2026SSG: Logit-Balanced Vocabulary Partitioning for LLM Watermarkingcited by 0