# CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Code arXiv: https://arxiv.org/abs/2404.15639 ยท HTML: https://mlsanity.com/paper/2404.15639/ ## Key facts - Cited by 24 papers in the map (data through 14 Oct 2026). - Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. - Builds on 25 of its 57 references that are in the map. - First posted on arXiv 24 Apr 2024. ## Placed in - LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/) ## Builds on (most cited first, 25 of 25) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019) - [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021) - [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021) - [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015) - [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023) - [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022) - [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019) - [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019) - [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020) - [code2seq: Generating Sequences from Structured Representations of Code](https://mlsanity.com/paper/1808.01400/) (2018) - [Who Wrote this Code? Watermarking for Code Generation](https://mlsanity.com/paper/2305.15060/) (2023) - [Improving Automatic Source Code Summarization via Deep Reinforcement Learning](https://mlsanity.com/paper/1811.07234/) (2018) - [Real or Fake? Learning to Discriminate Machine from Human Generated Text](https://mlsanity.com/paper/1906.03351/) (2019) - [Towards Codable Watermarking for Injecting Multi-bits Information to LLMs](https://mlsanity.com/paper/2307.15992/) (2023) - [LLM-as-a-Coauthor: Can Mixed Human-Written and Machine-Generated Text Be Detected?](https://mlsanity.com/paper/2401.05952/) (2024) - [Iterative Refinement of Project-Level Code Context for Precise Code Generation with Compiler Feedback](https://mlsanity.com/paper/2403.16792/) (2024) - [Multi-Modal Attention Network Learning for Semantic Source Code Retrieval](https://mlsanity.com/paper/1909.13516/) (2019) - [CodeMark: Imperceptible Watermarking for Code Datasets against Neural Code Completion Models](https://mlsanity.com/paper/2308.14401/) (2023) - [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023) - [Enhancing Code Generation Performance of Smaller Models by Distilling the Reasoning Ability of LLMs](https://mlsanity.com/paper/2403.13271/) (2024) - [Reverse Engineering Configurations of Neural Text Generation Models](https://mlsanity.com/paper/2004.06201/) (2020) - [IRCoCo: Immediate Rewards-Guided Deep Reinforcement Learning for Code Completion](https://mlsanity.com/paper/2401.16637/) (2024) - [Towards Code Watermarking with Dual-Channel Transformations](https://mlsanity.com/paper/2309.00860/) (2023) ## Built on by (most cited first, 24 of 24) - [Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2502.06872/) (2025) - [Marking Code Without Breaking It: Code Watermarking for Detecting LLM-Generated Code](https://mlsanity.com/paper/2502.18851/) (2025) - [Agent Guide: A Simple Agent Behavioral Watermarking Framework](https://mlsanity.com/paper/2504.05871/) (2025) - [Watermarking Language Models through Language Models](https://mlsanity.com/paper/2411.05091/) (2024) - [AgentMark: Utility-Preserving Behavioral Watermarking for Agents](https://mlsanity.com/paper/2601.03294/) (2026) - [SAEMark: Steering Personalized Multilingual LLM Watermarks with Sparse Autoencoders](https://mlsanity.com/paper/2508.08211/) (2025) - [Discovering Spoofing Attempts on Language Model Watermarks](https://mlsanity.com/paper/2410.02693/) (2024) - [Invisible Entropy: Towards Safe and Efficient Low-Entropy LLM Watermarking](https://mlsanity.com/paper/2505.14112/) (2025) - [WaterMod: Modular Token-Rank Partitioning for Probability-Balanced LLM Watermarking](https://mlsanity.com/paper/2511.07863/) (2025) - [XAttnMark: Learning Robust Audio Watermarking with Cross-Attention](https://mlsanity.com/paper/2502.04230/) (2025) - [From Trade-off to Synergy: A Versatile Symbiotic Watermarking Framework for Large Language Models](https://mlsanity.com/paper/2505.09924/) (2025) - [AI Code in the Wild: Measuring Security Risks and Ecosystem Shifts of AI-Generated Code in Modern Software](https://mlsanity.com/paper/2512.18567/) (2025) - [From Intentions to Techniques: A Comprehensive Taxonomy and Challenges in Text Watermarking for Large Language Models](https://mlsanity.com/paper/2406.11106/) (2024) - [Enhancing Watermarking Quality for LLMs via Contextual Generation States Awareness](https://mlsanity.com/paper/2506.07403/) (2025) - [SoK: Are Watermarks in LLMs Ready for Deployment?](https://mlsanity.com/paper/2506.05594/) (2025) - [MCGMark: An Encodable and Robust Online Watermark for Tracing LLM-Generated Malicious Code](https://mlsanity.com/paper/2408.01354/) (2024) - [Optimizing Token Choice for Code Watermarking: An RL Approach](https://mlsanity.com/paper/2508.11925/) (2025) - [Bridging Generation and Training: A Systematic Review of Quality Issues in LLMs for Code](https://mlsanity.com/paper/2605.05267/) (2026) - [Disappearing Ink: Obfuscation Breaks N-gram Code Watermarks in Theory and Practice](https://mlsanity.com/paper/2507.05512/) (2025) - [CLASP: Training-Free LLM-Assisted Source Code Watermarking via Semantic-Preserving Transformations](https://mlsanity.com/paper/2510.11251/) (2025) - [CODE ACROSTIC: Robust Watermarking for Code Generation](https://mlsanity.com/paper/2512.14753/) (2025) - [Multi-Channel Spread-Spectrum Code Watermarking](https://mlsanity.com/paper/2607.06009/) (2026) - [TrajMark: Ownership Attribution and Segment-Level Tamper Localization for Coding-Agent Trajectories](https://mlsanity.com/paper/2609.10416/) (2026) - [Grammar-Guided Code Watermarking with Green Temperature](https://mlsanity.com/paper/2610.05323/) (2026) ## Abstract (arXiv) Large Language Models (LLMs) have achieved remarkable progress in code generation. It now becomes crucial to identify whether the code is AI-generated and to determine the specific model used, particularly for purposes such as protecting Intellectual Property (IP) in industry and preventing cheating in programming exercises. To this end, several attempts have been made to insert watermarks into machine-generated code. However, existing approaches are limited to inserting only a single bit of information. In this paper, we introduce CodeIP, a novel multi-bit watermarking technique that inserts additional information to preserve crucial provenance details, such as the vendor ID of an LLM, thereby safeguarding the IPs of LLMs in code generation. Furthermore, to ensure the syntactical correctness of the generated code, we propose constraining the sampling process for predicting the next token by training a type predictor. Experiments conducted on a real-world dataset across five programming languages demonstrate the effectiveness of CodeIP in watermarking LLMs for code generation while maintaining the syntactical correctness of code.