mlsanityData to 7 Oct 2026

Watermarking Large Language Models and the Generated Content: Opportunities and Challenges

arXiv 2410.19096 ↗First posted 24 Oct 2024cs.CR cs.CLcited by 2

On the map: LLM safety and security → Watermarking generated content and models

Abstract (arXiv)

The widely adopted and powerful generative large language models (LLMs) have raised concerns about intellectual property rights violations and the spread of machine-generated misinformation. Watermarking serves as a promising approch to establish ownership, prevent unauthorized use, and trace the origins of LLM-generated content. This paper summarizes and shares the challenges and opportunities we found when watermarking LLMs. We begin by introducing techniques for watermarking LLMs themselves under different threat models and scenarios. Next, we investigate watermarking methods designed for the content generated by LLMs, assessing their effectiveness and resilience against various attacks. We also highlight the importance of watermarking domain-specific models and data, such as those used in code generation, chip design, and medical applications. Furthermore, we explore methods like hardware acceleration to improve the efficiency of the watermarking process. Finally, we discuss the limitations of current approaches and outline future research directions for the responsible use and protection of these generative AI tools.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. Builds on 64 of its 105 references that are in the map. First posted on arXiv 24 Oct 2024.

Builds on

64 of 105 references.

Closest ancestors · 21

Same direction, newest first

  1. 2024Explanation as a Watermark: Towards Harmless and Multi-bit Model Ownership Verification via Watermarking Feature Attribution8 May
  2. 2024Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models28 Feb
  3. 2024EmMark: Robust Watermarks for IP Protection of Embedded Quantized Large Language Models27 Feb
  4. 2024Double-I Watermark: Protecting Model Copyright for LLM Fine-tuning22 Feb
  5. 2024Proving membership in LLM pretraining data via data watermarks16 Feb
  6. 2024Adaptive Text Watermark for Large Language Models25 Jan
  7. 2023REMARK-LLM: A Robust and Efficient Watermarking Framework for Generative Large Language Models18 Oct
  8. 2023Watermarking LLMs with Weight Quantization17 Oct
  9. 2023ZKROWNN: Zero Knowledge Right of Ownership for Neural Networks13 Sep
  10. 2023CodeMark: Imperceptible Watermarking for Code Datasets against Neural Code Completion Models28 Aug
  11. 2023Robust Distortion-free Watermarks for Language Models28 Jul
  12. 2023Provable Robust Watermarking for AI-Generated Text30 Jun
9 more
  1. 2023On the Reliability of Watermarks for Large Language Models7 Jun
  2. 2023Undetectable Watermarks for Language Models25 May
  3. 2023Who Wrote this Code? Watermarking for Code Generation24 May
  4. 2023Watermarking Text Generated by Black-Box Language Models14 May
  5. 2023Robust Multi-bit Natural Language Watermarking through Invariant Features3 May
  6. 2023A Watermark for Large Language Models24 Jan
  7. 2022Watermarking Pre-trained Language Models with Backdooring14 Oct
  8. 2022CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks19 Sep
  9. 2021Protecting Intellectual Property of Language Generation APIs with Lexical Watermark5 Dec

Other references · 43

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  4. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  5. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  6. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  7. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  8. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  9. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  10. 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 400
  11. 2024Mixture-of-Depths: Dynamically allocating compute in transformer-based language modelscited by 242
  12. 2023Using an LLM to Help With Code Understandingcited by 222
31 more
  1. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 200
  2. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  3. 2023ChipNeMo: Domain-Adapted LLMs for Chip Designcited by 168
  4. 2023Speculative Decoding with Big Little Decodercited by 163
  5. 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 160
  6. 2023The Science of Detecting LLM-Generated Textscited by 126
  7. 2023BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questionscited by 125
  8. 2020Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hidingcited by 121
  9. 2024Not All Layers of LLMs Are Necessary During Inferencecited by 95
  10. 2023Combating Misinformation in the Age of LLMs: Opportunities and Challengescited by 94
  11. 2023AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Datasetcited by 79
  12. 2024Instructional Fingerprinting of Large Language Modelscited by 64
  13. 2024FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAscited by 60
  14. 2020A Systematic Review on Model Watermarking for Neural Networkscited by 49
  15. 2023Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiencycited by 46
  16. 2024A Safe Harbor for AI Evaluation and Red Teamingcited by 46
  17. 2024LLM Inference Serving: Survey of Recent Advances and Opportunitiescited by 46
  18. 2024A Survey on Generative AI and LLM for Video Generation, Understanding, and Streamingcited by 37
  19. 2024JMLR: Joint Medical LLM and Retrieval Training for Enhancing Reasoning and Professional Question Answering Capabilitycited by 32
  20. 2024A Continued Pretrained LLM Approach for Automatic Medical Note Generationcited by 30
  21. 2024Rome was Not Built in a Single Step: Hierarchical Prompting for LLM-based Chip Designcited by 24
  22. 2024Evaluating LLM -- Generated Multimodal Diagnosis from Medical Images and Symptom Analysiscited by 23
  23. 2024EdgeLLM: A Highly Efficient CPU-FPGA Heterogeneous Edge Accelerator for Large Language Modelscited by 19
  24. 2024From Data to Story: Towards Automatic Animated Data Video Creation with LLM-based Multi-Agent Systemscited by 16
  25. 2023Towards Code Watermarking with Dual-Channel Transformationscited by 9
  26. 2024A Fast, Performant, Secure Distributed Training Framework For Large Language Modelcited by 8
  27. 2024A First Look At Efficient And Secure On-Device LLM Inference Against KV Leakagecited by 8
  28. 2024Model Parallelism on Distributed Infrastructure: A Literature Review from Theory to LLM Case-Studiescited by 7
  29. 2022FastStamp: Accelerating Neural Steganography and Digital Watermarking of Images on FPGAscited by 2
  30. 2024ICMarks: A Robust Watermarking Framework for Integrated Circuit Physical Design IP Protectioncited by 2
  31. 2024Automated Physical Design Watermarking Leveraging Graph Neural Networkscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025A Comprehensive Dataset for Human vs. AI Generated Text Detectioncited by 3
  2. 2026Flip, Don't Shuffle: Watermarking LLMs at the Speed of Inferencecited by 0