mlsanityData to 7 Oct 2026

Topic-Based Watermarks for Large Language Models

arXiv 2404.02138 ↗First posted 2 Apr 2024cs.CR cs.CL cs.LGcited by 7

On the map: LLM safety and security → Watermarking generated content and models

Abstract (arXiv)

The indistinguishability of large language model (LLM) output from human-authored content poses significant challenges, raising concerns about potential misuse of AI-generated text and its influence on future model training. Watermarking algorithms offer a viable solution by embedding detectable signatures into generated text. However, existing watermarking methods often involve trade-offs among attack robustness, generation quality, and additional overhead such as specialized frameworks or complex integrations. We propose a lightweight, topic-guided watermarking scheme for LLMs that partitions the vocabulary into topic-aligned token subsets. Given an input prompt, the scheme selects a relevant topic-specific token list, effectively "green-listing" semantically aligned tokens to embed robust marks while preserving fluency and coherence. Experimental results across multiple LLMs and state-of-the-art benchmarks demonstrate that our method achieves text quality comparable to industry-leading systems and simultaneously improves watermark robustness against paraphrasing and lexical perturbation attacks, with minimal performance overhead. Our approach avoids reliance on additional mechanisms beyond standard text generation pipelines, enabling straightforward adoption and suggesting a practical path toward globally consistent watermarking of AI-generated content.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. Builds on 28 of its 54 references that are in the map. First posted on arXiv 2 Apr 2024.

Builds on

28 of 54 references.

Closest ancestors · 16

Same direction, newest first

  1. 2024Watermark Stealing in Large Language Models29 Feb
  2. 2024Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models28 Feb
  3. 2024k-SemStamp: A Clustering-Based Semantic Watermark for Detection of Machine-Generated Text17 Feb
  4. 2024Provably Robust Multi-bit Watermarking for AI-generated Text30 Jan
  5. 2024Adaptive Text Watermark for Large Language Models25 Jan
  6. 2023REMARK-LLM: A Robust and Efficient Watermarking Framework for Generative Large Language Models18 Oct
  7. 2023Embarrassingly Simple Text Watermarks13 Oct
  8. 2023A Resilient and Accessible Distribution-Preserving Watermark for Large Language Models11 Oct
  9. 2023A Semantic Invariant Robust Watermark for Large Language Models10 Oct
  10. 2023Robust Distortion-free Watermarks for Language Models28 Jul
  11. 2023Provable Robust Watermarking for AI-Generated Text30 Jun
  12. 2023Who Wrote this Code? Watermarking for Code Generation24 May
4 more
  1. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense23 Mar
  2. 2023Can AI-Generated Text be Reliably Detected?17 Mar
  3. 2023Protecting Language Generation Models via Invisible Watermarking6 Feb
  4. 2023A Watermark for Large Language Models24 Jan

Other references · 12

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  5. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
  6. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 705
  7. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  8. 2023GPT detectors are biased against non-native English writerscited by 175
  9. 2020TLDR: Extreme Summarization of Scientific Documentscited by 136
  10. 2023Combating Misinformation in the Age of LLMs: Opportunities and Challengescited by 94
  11. 2022Do Language Models Plagiarize?cited by 72
  12. 2023Distinguishing Human Generated Text From ChatGPT Generated Text Using Machine Learningcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.