mlsanityData to 7 Oct 2026

Who Wrote this Code? Watermarking for Code Generation

arXiv 2305.15060 ↗First posted 24 May 2023cs.CLcited by 116

On the map: LLM safety and security → Watermarking generated content and models

Abstract (arXiv)

Since the remarkable generation performance of large language models raised ethical and legal concerns, approaches to detect machine-generated text by embedding watermarks are being developed. However, we discover that the existing works fail to function appropriately in code generation tasks due to the task's nature of having low entropy. Extending a logit-modifying watermark method, we propose Selective WatErmarking via Entropy Thresholding (SWEET), which enhances detection ability and mitigates code quality degeneration by removing low-entropy segments at generating and detecting watermarks. Our experiments show that SWEET significantly improves code quality preservation while outperforming all baselines, including post-hoc detection methods, in detecting machine-generated code text. Our code is available in https://github.com/hongcheki/sweet-watermark.

Key facts. Cited by 116 papers in the map (data through 7 Oct 2026). Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. Builds on 36 of its 74 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

36 of 74 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense23 Mar
  2. 2023Can AI-Generated Text be Reliably Detected?17 Mar
  3. 2023A Watermark for Large Language Models24 Jan
  4. 2022CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks19 Sep
  5. 2021Tracing Text Provenance via Context-Aware Lexical Substitution15 Dec
  6. 2021Protecting Intellectual Property of Language Generation APIs with Lexical Watermark5 Dec

Other references · 30

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2021Program Synthesis with Large Language Modelscited by 3,509
  9. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  10. 2020Extracting Training Data from Large Language Modelscited by 2,144
  11. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  12. 2023StarCoder: may the source be with you!cited by 790
18 more
  1. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 705
  2. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  3. 2019Defending Against Neural Fake Newscited by 553
  4. 2019Release Strategies and the Social Impacts of Language Modelscited by 505
  5. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  6. 2019GLTR: Statistical Detection and Visualization of Generated Textcited by 490
  7. 2023How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detectioncited by 472
  8. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
  9. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 417
  10. 2019Automatic Detection of Generated Text is Easiest when Humans are Fooledcited by 260
  11. 2022Efficient Training of Language Models to Fill in the Middlecited by 248
  12. 2023SantaCoder: don't reach for the stars!cited by 169
  13. 2023Large Language Models for Code: Security Hardening and Adversarial Testingcited by 149
  14. 2020Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hidingcited by 121
  15. 2022Lost at C: A User Study on the Security Implications of Large Language Model Code Assistantscited by 87
  16. 2023ChatGPT or Human? Detect and Explain. Explaining Decisions of Machine Learning Model for Detecting Short ChatGPT-generated Textcited by 84
  17. 2023Origin Tracing and Detecting of LLMscited by 39
  18. 2021The Threat of Offensive AI to Organizationscited by 37

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 28; 2025: 47; 2026: 26.

116 papers.

Sort:

By area:

  1. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  2. 2023On the Reliability of Watermarks for Large Language Modelscited by 170
  3. 2023A Survey of Text Watermarking in the Era of Large Language Modelscited by 102
  4. 2023A Semantic Invariant Robust Watermark for Large Language Modelscited by 97
  5. 2024MarkLLM: An Open-Source Toolkit for LLM Watermarkingcited by 80
  6. 2023A Survey on Detection of LLMs-Generated Contentcited by 70
  7. 2023Mark My Words: Analyzing and Evaluating Language Model Watermarkscited by 62
  8. 2024An Entropy-based Text Watermarking Detection Methodcited by 61
  9. 2024Adaptive Text Watermark for Large Language Modelscited by 60
  10. 2023An Unforgeable Publicly Verifiable Watermark for Large Language Modelscited by 54
  11. 2024Watermarking Makes Language Models Radioactivecited by 52
  12. 2023Advancing Beyond Identification: Multi-bit Watermark for Large Language Modelscited by 51
38 more
  1. 2023Watermarking Conditional Text Generation for AI Detection: Unveiling Challenges and a Semantic-Aware Watermark Remedycited by 45
  2. 2023Towards Codable Watermarking for Injecting Multi-bits Information to LLMscited by 43
  3. 2024SoK: Watermarking for AI-Generated Contentcited by 43
  4. 2024Securing Large Language Models: Threats, Vulnerabilities and Responsible Practicescited by 36
  5. 2024Copyright Protection in Generative AI: A Technical Perspectivecited by 35
  6. 2024Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Modelscited by 30
  7. 2024Between Lines of Code: Unraveling the Distinct Patterns of Machine and Human Programmerscited by 27
  8. 2023BioCoder: A Benchmark for Bioinformatics Code Generation with Large Language Modelscited by 24
  9. 2023Zero-Shot Detection of Machine-Generated Codescited by 24
  10. 2023Grounding Foundation Models through Federated Transfer Learning: A General Frameworkcited by 24
  11. 2023Optimizing watermarks for large language modelscited by 22
  12. 2024CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Codecited by 21
  13. 2025Self-Evolving Multi-Agent Systems via Textual Backpropagationcited by 21
  14. 2024Watermarking Techniques for Large Language Models: A Surveycited by 20
  15. 2024Learning to Watermark LLM-generated Text via Reinforcement Learningcited by 19
  16. 2025Can LLM Watermarks Robustly Prevent Unauthorized Knowledge Distillation?cited by 16
  17. 2024A Watermark for Black-Box Language Modelscited by 14
  18. 2025Mark Your LLM: Detecting the Misuse of Open-Source Large Language Models via Watermarkingcited by 14
  19. 2024Large Language Model Watermark Stealing With Mixed Integer Programmingcited by 13
  20. 2024A Practical Examination of AI-Generated Text Detectors for Large Language Modelscited by 11
  21. 2025VLA-Mark: A cross modal watermark for large vision-language alignment modelcited by 11
  22. 2025Defending LLM Watermarking Against Spoofing Attacks with Contrastive Representation Learningcited by 10
  23. 2024WaterSeeker: Pioneering Efficient Detection of Watermarked Segments in Large Documentscited by 9
  24. 2025In-Context Watermarks for Large Language Modelscited by 9
  25. 2023Assaying on the Robustness of Zero-Shot Machine-Generated Text Detectorscited by 8
  26. 2024Duwak: Dual Watermarks in Large Language Modelscited by 8
  27. 2025DuFFin: A Dual-Level Fingerprinting Framework for LLMs IP Protectioncited by 8
  28. 2025Enhancing LLM Watermark Resilience Against Both Scrubbing and Spoofing Attackscited by 8
  29. 2024Topic-Based Watermarks for Large Language Modelscited by 7
  30. 2024Watermark under Fire: A Robustness Evaluation of LLM Watermarkingcited by 7
  31. 2025Marking Code Without Breaking It: Code Watermarking for Detecting LLM-Generated Codecited by 7
  32. 2025Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoptioncited by 7
  33. 2024PersonaMark: Personalized LLM watermarking for model protection and user attributioncited by 6
  34. 2025Detecting Benchmark Contamination Through Watermarkingcited by 6
  35. 2025From Trade-off to Synergy: A Versatile Symbiotic Watermarking Framework for Large Language Modelscited by 5
  36. 2025Invisible Entropy: Towards Safe and Efficient Low-Entropy LLM Watermarkingcited by 5
  37. 2025SAEMark: Steering Personalized Multilingual LLM Watermarks with Sparse Autoencoderscited by 5
  38. 2026AgentMark: Utility-Preserving Behavioral Watermarking for Agentscited by 5

50 most cited of 116 All 116 (JSON)