mlsanityData to 14 Oct 2026

A Watermark for Order-Agnostic Language Models

arXiv 2410.13805 ↗First posted 17 Oct 2024cs.CLcited by 14

On the map: LLM safety and security → Watermarking generated content and models

Abstract (arXiv)

Statistical watermarking techniques are well-established for sequentially decoded language models (LMs). However, these techniques cannot be directly applied to order-agnostic LMs, as the tokens in order-agnostic LMs are not generated sequentially. In this work, we introduce Pattern-mark, a pattern-based watermarking framework specifically designed for order-agnostic LMs. We develop a Markov-chain-based watermark generator that produces watermark key sequences with high-frequency key patterns. Correspondingly, we propose a statistical pattern-based detection algorithm that recovers the key sequence during detection and conducts statistical tests based on the count of high-frequency patterns. Our extensive evaluations on order-agnostic LMs, such as ProteinMPNN and CMLM, demonstrate Pattern-mark's enhanced detection efficiency, generation quality, and robustness, positioning it as a superior watermarking technique for order-agnostic LMs.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. Builds on 21 of its 45 references that are in the map. First posted on arXiv 17 Oct 2024.

Builds on

21 of 45 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024Distortion-free Watermarks are not Truly Distortion-free under Watermark Key Collisions2 Jun
  2. 2024Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models28 Feb
  3. 2023REMARK-LLM: A Robust and Efficient Watermarking Framework for Generative Large Language Models18 Oct
  4. 2023A Semantic Invariant Robust Watermark for Large Language Models10 Oct
  5. 2023Unbiased Watermark for Large Language Models22 Sep
  6. 2023An Unforgeable Publicly Verifiable Watermark for Large Language Models30 Jul
  7. 2023Robust Distortion-free Watermarks for Language Models28 Jul
  8. 2023Provable Robust Watermarking for AI-Generated Text30 Jun
  9. 2023On the Reliability of Watermarks for Large Language Models7 Jun
  10. 2023Undetectable Watermarks for Language Models25 May
  11. 2023A Watermark for Large Language Models24 Jan

Other references · 10

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2017Non-Autoregressive Neural Machine Translationcited by 463
  4. 2024Your Vision-Language Model Itself Is a Strong Filter: Towards High-Quality Instruction Tuning with Data Selectioncited by 48
  5. 2022Adversarial Weight Perturbation Improves Generalization in Graph Neural Networkscited by 21
  6. 2023Learning to Learn from APIs: Black-Box Data-Free Meta-Learningcited by 18
  7. 2022RetrievalGuard: Provably Robust 1-Nearest Neighbor Image Retrievalcited by 15
  8. 2024Few-Shot Class Incremental Learning with Attention-Aware Self-Adaptive Promptcited by 15
  9. 2022A Law of Robustness beyond Isoperimetrycited by 9
  10. 2021NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecastingcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 7.

14 papers.

Sort:

By area:

  1. 2025AdaDetectGPT: Adaptive Detection of LLM-Generated Text with Statistical Guaranteescited by 21
  2. 2025Improved Unbiased Watermark for Large Language Modelscited by 17
  3. 2025In-Context Watermarks for Large Language Modelscited by 10
  4. 2025Watermarking Diffusion Language Modelscited by 10
  5. 2025Toward Robust Non-Transferable Learning: A Survey and Benchmarkcited by 7
  6. 2025Explore Data Left Behind in Reinforcement Learning for Reasoning Language Modelscited by 7
  7. 2025Detecting Post-generation Edits to Watermarked LLM Outputs via Combinatorial Watermarkingcited by 3
  8. 2026More Haste, Less Speed: Weaker Single-Layer Watermark Improves Distortion-Free Watermark Ensemblescited by 2
  9. 2026SAMark: A Self-Anchored Text Watermarking with Paragraph-Level Paraphrase Robustnesscited by 2
  10. 2026Chainwash: Multi-Step Rewriting Attacks on Diffusion Language Model Watermarkscited by 1
  11. 2026AliMark: Enhancing Robustness of Sentence-Level Watermarking Against Text Paraphrasingcited by 1
  12. 2026SAC-Copula: Quality-Preserving Watermarking for Diffusion Language Models via Smooth Correlated Gumbel Fieldscited by 1
2 more
  1. 2026DenMark: Robust Semantic Watermarking for Diffusion Language Modelscited by 1
  2. 2026Anchor-ECC: Local Integrity Checking for Watermarked LLM Outputs via Error-Correcting Codescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Watermark for Black-Box Language Models
  2. 2024A Statistical Framework of Watermarks for Large Language Models: Pivot, Detection Efficiency and Optimal Rules
  3. 2026ChainMark: Model-Free LLM Watermarking with Closed-Form Calibration
  4. 2024Task-Agnostic Language Model Watermarking via High Entropy Passthrough Layers
  5. 2026Flip, Don't Shuffle: Watermarking LLMs at the Speed of Inference
  6. 2024Black-Box Detection of Language Model Watermarks
  7. 2024Learnable Linguistic Watermarks for Tracing Model Extraction Attacks on Large Language Models
  8. 2024Watermarking Low-entropy Generation for Large Language Models: An Unbiased and Low-risk Method