# MarkLLM: An Open-Source Toolkit for LLM Watermarking

arXiv: https://arxiv.org/abs/2405.10051 · HTML: https://mlsanity.com/paper/2405.10051/

## Key facts

- Cited by 95 papers in the map (data through 14 Oct 2026).
- Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security.
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 16 May 2024.

## Placed in

- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Large Language Models for Information Retrieval: A Survey](https://mlsanity.com/paper/2308.07107/) (2023)
- [Robust Distortion-free Watermarks for Language Models](https://mlsanity.com/paper/2307.15593/) (2023)
- [Provable Robust Watermarking for AI-Generated Text](https://mlsanity.com/paper/2306.17439/) (2023)
- [In-Context Impersonation Reveals Large Language Models' Strengths and Biases](https://mlsanity.com/paper/2305.14930/) (2023)
- [Who Wrote this Code? Watermarking for Code Generation](https://mlsanity.com/paper/2305.15060/) (2023)
- [A Survey of Text Watermarking in the Era of Large Language Models](https://mlsanity.com/paper/2312.07913/) (2023)
- [A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writing](https://mlsanity.com/paper/2310.08433/) (2023)
- [A Robust Semantics-based Watermark for Large Language Model against Paraphrasing](https://mlsanity.com/paper/2311.08721/) (2023)
- [On the Learnability of Watermarks for Language Models](https://mlsanity.com/paper/2312.04469/) (2023)
- [Publicly-Detectable Watermarking for Language Models](https://mlsanity.com/paper/2310.18491/) (2023)
- [An Entropy-based Text Watermarking Detection Method](https://mlsanity.com/paper/2403.13485/) (2024)
- [Mark My Words: Analyzing and Evaluating Language Model Watermarks](https://mlsanity.com/paper/2312.00273/) (2023)
- [Towards Codable Watermarking for Injecting Multi-bits Information to LLMs](https://mlsanity.com/paper/2307.15992/) (2023)
- [HowkGPT: Investigating the Detection of ChatGPT-generated University Student Homework through Context-Aware Perplexity Analysis](https://mlsanity.com/paper/2305.18226/) (2023)

## Built on by (most cited first, 30 of 95)

- [SoK: Watermarking for AI-Generated Content](https://mlsanity.com/paper/2411.18479/) (2024)
- [Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2502.06872/) (2025)
- [MorphMark: Flexible Adaptive Watermarking for Large Language Models](https://mlsanity.com/paper/2505.11541/) (2025)
- [Watermarking Techniques for Large Language Models: A Survey](https://mlsanity.com/paper/2409.00089/) (2024)
- [Frontier AI's Impact on the Cybersecurity Landscape](https://mlsanity.com/paper/2504.05408/) (2025)
- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [Can LLM Watermarks Robustly Prevent Unauthorized Knowledge Distillation?](https://mlsanity.com/paper/2502.11598/) (2025)
- [Mark Your LLM: Detecting the Misuse of Open-Source Large Language Models via Watermarking](https://mlsanity.com/paper/2503.04636/) (2025)
- [CoheMark: A Novel Sentence-Level Watermark for Enhanced Text Quality](https://mlsanity.com/paper/2504.17309/) (2025)
- [Revealing Weaknesses in Text Watermarking Through Self-Information Rewrite Attacks](https://mlsanity.com/paper/2505.05190/) (2025)
- [Optimizing Adaptive Attacks against Watermarks for Language Models](https://mlsanity.com/paper/2410.02440/) (2024)
- [PMark: Towards Robust and Distortion-free Semantic-level Watermarking with Channel Constraints](https://mlsanity.com/paper/2509.21057/) (2025)
- [SimMark: A Robust Sentence-Level Similarity-Based Watermarking Algorithm for Large Language Models](https://mlsanity.com/paper/2502.02787/) (2025)
- [Can Watermarked LLMs be Identified by Users via Crafted Prompts?](https://mlsanity.com/paper/2410.03168/) (2024)
- [Dataset Protection via Watermarked Canaries in Retrieval-Augmented LLMs](https://mlsanity.com/paper/2502.10673/) (2025)
- [Unified attacks to large language model watermarks: spoofing and scrubbing in unauthorized knowledge distillation](https://mlsanity.com/paper/2504.17480/) (2025)
- [Agent Guide: A Simple Agent Behavioral Watermarking Framework](https://mlsanity.com/paper/2504.05871/) (2025)
- [In-Context Watermarks for Large Language Models](https://mlsanity.com/paper/2505.16934/) (2025)
- [Watermarking Diffusion Language Models](https://mlsanity.com/paper/2509.24368/) (2025)
- [WaterSeeker: Pioneering Efficient Detection of Watermarked Segments in Large Documents](https://mlsanity.com/paper/2409.05112/) (2024)
- [Optimized Couplings for Watermarking Large Language Models](https://mlsanity.com/paper/2505.08878/) (2025)
- [Watermarking Without Standards Is Not AI Governance](https://mlsanity.com/paper/2505.23814/) (2025)
- [Character-Level Perturbations Disrupt LLM Watermarks](https://mlsanity.com/paper/2509.09112/) (2025)
- [SAEMark: Steering Personalized Multilingual LLM Watermarks with Sparse Autoencoders](https://mlsanity.com/paper/2508.08211/) (2025)
- [Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoption](https://mlsanity.com/paper/2510.18333/) (2025)
- [Invisible Entropy: Towards Safe and Efficient Low-Entropy LLM Watermarking](https://mlsanity.com/paper/2505.14112/) (2025)
- [From Trade-off to Synergy: A Versatile Symbiotic Watermarking Framework for Large Language Models](https://mlsanity.com/paper/2505.09924/) (2025)
- [Factuality Beyond Coherence: Evaluating LLM Watermarking Methods for Medical Texts](https://mlsanity.com/paper/2509.07755/) (2025)
- [MarkDiffusion: An Open-Source Toolkit for Generative Watermarking of Latent Diffusion Models](https://mlsanity.com/paper/2509.10569/) (2025)
- [Analyzing and Evaluating Unbiased Language Model Watermark](https://mlsanity.com/paper/2509.24048/) (2025)

## Abstract (arXiv)

LLM watermarking, which embeds imperceptible yet algorithmically detectable signals in model outputs to identify LLM-generated text, has become crucial in mitigating the potential misuse of large language models. However, the abundance of LLM watermarking algorithms, their intricate mechanisms, and the complex evaluation procedures and perspectives pose challenges for researchers and the community to easily experiment with, understand, and assess the latest advancements. To address these issues, we introduce MarkLLM, an open-source toolkit for LLM watermarking. MarkLLM offers a unified and extensible framework for implementing LLM watermarking algorithms, while providing user-friendly interfaces to ensure ease of access. Furthermore, it enhances understanding by supporting automatic visualization of the underlying mechanisms of these algorithms. For evaluation, MarkLLM offers a comprehensive suite of 12 tools spanning three perspectives, along with two types of automated evaluation pipelines. Through MarkLLM, we aim to support researchers while improving the comprehension and involvement of the general public in LLM watermarking technology, fostering consensus and driving further advancements in research and application. Our code is available at https://github.com/THU-BPM/MarkLLM.