mlsanityData to 14 Oct 2026

De-mark: Watermark Removal in Large Language Models

arXiv 2410.13808 ↗First posted 17 Oct 2024cs.CLcited by 14

On the map: LLM safety and security → Watermarking generated content and models

Abstract (arXiv)

Watermarking techniques offer a promising way to identify machine-generated content via embedding covert information into the contents generated from language models (LMs). However, the robustness of the watermarking schemes has not been well explored. In this paper, we present De-mark, an advanced framework designed to remove n-gram-based watermarks effectively. Our method utilizes a novel querying strategy, termed random selection probing, which aids in assessing the strength of the watermark and identifying the red-green list within the n-gram watermark. Experiments on popular LMs, such as Llama3 and ChatGPT, demonstrate the efficiency and effectiveness of De-mark in watermark removal and exploitation tasks.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. Builds on 32 of its 52 references that are in the map. First posted on arXiv 17 Oct 2024.

Builds on

32 of 52 references.

Closest ancestors · 15

Same direction, newest first

  1. 2024Can Watermarked LLMs be Identified by Users via Crafted Prompts?4 Oct
  2. 2024Distortion-free Watermarks are not Truly Distortion-free under Watermark Key Collisions2 Jun
  3. 2024Large Language Model Watermark Stealing With Mixed Integer Programming30 May
  4. 2024Black-Box Detection of Language Model Watermarks28 May
  5. 2024Watermark Stealing in Large Language Models29 Feb
  6. 2024Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models28 Feb
  7. 2023Mark My Words: Analyzing and Evaluating Language Model Watermarks1 Dec
  8. 2023A Semantic Invariant Robust Watermark for Large Language Models10 Oct
  9. 2023Unbiased Watermark for Large Language Models22 Sep
  10. 2023Towards Codable Watermarking for Injecting Multi-bits Information to LLMs29 Jul
  11. 2023Robust Distortion-free Watermarks for Language Models28 Jul
  12. 2023Provable Robust Watermarking for AI-Generated Text30 Jun
3 more
  1. 2023On the Reliability of Watermarks for Large Language Models7 Jun
  2. 2023Undetectable Watermarks for Language Models25 May
  3. 2023A Watermark for Large Language Models24 Jan

Other references · 17

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Mistral 7Bcited by 3,164
  3. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  4. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  5. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
  6. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 501
  7. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  8. 2023Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questionscited by 263
  9. 2024Image Watermarks are Removable Using Controllable Regeneration from Clean Noisecited by 50
  10. 2024Your Vision-Language Model Itself Is a Strong Filter: Towards High-Quality Instruction Tuning with Data Selectioncited by 48
  11. 2023Split-and-Denoise: Protect large language model inference with local differential privacycited by 32
  12. 2022Adversarial Weight Perturbation Improves Generalization in Graph Neural Networkscited by 21
5 more
  1. 2023Learning to Learn from APIs: Black-Box Data-Free Meta-Learningcited by 18
  2. 2022RetrievalGuard: Provably Robust 1-Nearest Neighbor Image Retrievalcited by 15
  3. 2024Few-Shot Class Incremental Learning with Attention-Aware Self-Adaptive Promptcited by 15
  4. 2022A Law of Robustness beyond Isoperimetrycited by 9
  5. 2023ConfusionPrompt: Practical Private Inference for Online Large Language Modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 8.

14 papers.

Sort:

By area:

  1. 2025Character-Level Perturbations Disrupt LLM Watermarkscited by 9
  2. 2025An Ensemble Framework for Unbiased Language Model Watermarkingcited by 6
  3. 2025Analyzing and Evaluating Unbiased Language Model Watermarkcited by 5
  4. 2025DMark: Order-Agnostic Watermarking for Diffusion Large Language Modelscited by 4
  5. 2026A Visual Semantic Adaptive Watermark grounded by Prefix-Tuning for Large Vision-Language Modelcited by 3
  6. 2026Towards Robust Content Watermarking Against Removal and Forgery Attackscited by 3
  7. 2026RLSpoofer: A Lightweight Evaluator for LLM Watermark Spoofing Resiliencecited by 2
  8. 2026SIF: Semantically In-Distribution Fingerprints for Large Vision-Language Modelscited by 2
  9. 2025LLM Watermark Evasion via Bias Inversioncited by 1
  10. 2026Semantic Watermarking with Order-Robust Detection over Sub-sentence Unitscited by 1
  11. 2025Breaking Distortion-free Watermarks in Large Language Modelscited by 0
  12. 2026GoCoMA: Hyperbolic Multimodal Representation Fusion for Large Language Model-Generated Code Attributioncited by 0
2 more
  1. 2026A Regulatory Placebo? The Systemic Failure of Mandatory GenAI Labelingcited by 0
  2. 2026TANGO: Watermarking Masked Diffusion Language Models in Token Pairscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Baselines for Identifying Watermarked Large Language Models
  2. 2024Duwak: Dual Watermarks in Large Language Models
  3. 2024A Statistical Framework of Watermarks for Large Language Models: Pivot, Detection Efficiency and Optimal Rules
  4. 2025MorphMark: Flexible Adaptive Watermarking for Large Language Models
  5. 2023Three Bricks to Consolidate Watermarks for Large Language Models
  6. 2024Can Watermarks Survive Translation? On the Cross-lingual Consistency of Text Watermark for Large Language Models
  7. 2026Vaporizer: Breaking Watermarking Schemes for Large Language Model Outputs
  8. 2026Signature filtering: a lightweight enhancement for statistical watermark detection in large language models