# WAPITI: A Watermark for Finetuned Open-Source LLMs arXiv: https://arxiv.org/abs/2410.06467 ยท HTML: https://mlsanity.com/paper/2410.06467/ ## Key facts - Cited by 12 papers in the map (data through 14 Oct 2026). - Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. - Builds on 50 of its 67 references that are in the map. - First posted on arXiv 9 Oct 2024. ## Placed in - LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/) ## Builds on (most cited first, 30 of 50) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019) - [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023) - [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021) - [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020) - [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023) - [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022) - [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021) - [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023) - [Safe RLHF: Safe Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2310.12773/) (2023) - [Improving alignment of dialogue agents via targeted human judgements](https://mlsanity.com/paper/2209.14375/) (2022) - [Neural Text Generation with Unlikelihood Training](https://mlsanity.com/paper/1908.04319/) (2019) - [Memory-Based Model Editing at Scale](https://mlsanity.com/paper/2206.06520/) (2022) - [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023) - [Robust Distortion-free Watermarks for Language Models](https://mlsanity.com/paper/2307.15593/) (2023) - [Provable Robust Watermarking for AI-Generated Text](https://mlsanity.com/paper/2306.17439/) (2023) - [Undetectable Watermarks for Language Models](https://mlsanity.com/paper/2306.09194/) (2023) - [Training Neural Networks with Fixed Sparse Masks](https://mlsanity.com/paper/2111.09839/) (2021) - [Patching open-vocabulary models by interpolating weights](https://mlsanity.com/paper/2208.05592/) (2022) - [Automatic Detection of Machine Generated Text: A Critical Survey](https://mlsanity.com/paper/2011.01314/) (2020) - [Who Wrote this Code? Watermarking for Code Generation](https://mlsanity.com/paper/2305.15060/) (2023) - [Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hiding](https://mlsanity.com/paper/2009.03015/) (2020) - [A Survey of Text Watermarking in the Era of Large Language Models](https://mlsanity.com/paper/2312.07913/) (2023) - [Model Patching: Closing the Subgroup Performance Gap with Data Augmentation](https://mlsanity.com/paper/2008.06775/) (2020) - [A Robust Semantics-based Watermark for Large Language Model against Paraphrasing](https://mlsanity.com/paper/2311.08721/) (2023) - [On the Learnability of Watermarks for Language Models](https://mlsanity.com/paper/2312.04469/) (2023) - [Neural Linguistic Steganography](https://mlsanity.com/paper/1909.01496/) (2019) - [An Entropy-based Text Watermarking Detection Method](https://mlsanity.com/paper/2403.13485/) (2024) - [A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning](https://mlsanity.com/paper/2307.09218/) (2023) - [Adaptive Text Watermark for Large Language Models](https://mlsanity.com/paper/2401.13927/) (2024) ## Built on by (most cited first, 12 of 12) - [SelfElicit: Your Language Model Secretly Knows Where is the Relevant Evidence](https://mlsanity.com/paper/2502.08767/) (2025) - [Towards Watermarking of Open-Source LLMs](https://mlsanity.com/paper/2502.10525/) (2025) - [Ask, and it shall be given: On the Turing completeness of prompting](https://mlsanity.com/paper/2411.01992/) (2024) - [AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism](https://mlsanity.com/paper/2506.03700/) (2025) - [SAEMark: Steering Personalized Multilingual LLM Watermarks with Sparse Autoencoders](https://mlsanity.com/paper/2508.08211/) (2025) - [AdaFuse: Adaptive Ensemble Decoding with Test-Time Scaling for LLMs](https://mlsanity.com/paper/2601.06022/) (2026) - [Self-Error-Instruct: Generalizing from Errors for LLMs Mathematical Reasoning](https://mlsanity.com/paper/2505.22591/) (2025) - [Saffron-1: Safety Inference Scaling](https://mlsanity.com/paper/2506.06444/) (2025) - [Graph Homophily Booster: Rethinking the Role of Discrete Features on Heterophilic Graphs](https://mlsanity.com/paper/2509.12530/) (2025) - [Graph homophily booster: Reimagining the role of discrete features in heterophilic graph learning](https://mlsanity.com/paper/2602.07256/) (2026) - [AvAtar: Learning to Align via Active Optimal Transport](https://mlsanity.com/paper/2605.24395/) (2026) - [Structuring and Tokenizing Distributed User Interest Context for Generative Recommendation](https://mlsanity.com/paper/2606.20554/) (2026) ## Abstract (arXiv) Watermarking of large language models (LLMs) generation embeds an imperceptible statistical pattern within texts, making it algorithmically detectable. Watermarking is a promising method for addressing potential harm and biases from LLMs, as it enables traceability, accountability, and detection of manipulated content, helping to mitigate unintended consequences. However, for open-source models, watermarking faces two major challenges: (i) incompatibility with fine-tuned models, and (ii) vulnerability to fine-tuning attacks. In this work, we propose WAPITI, a new method that transfers watermarking from base models to fine-tuned models through parameter integration. To the best of our knowledge, we propose the first watermark for fine-tuned open-source LLMs that preserves their fine-tuned capabilities. Furthermore, our approach offers an effective defense against fine-tuning attacks. We test our method on various model architectures and watermarking strategies. Results demonstrate that our method can successfully inject watermarks and is highly compatible with fine-tuned models. Additionally, we offer an in-depth analysis of how parameter editing influences the watermark strength and overall capabilities of the resulting models.