mlsanityData to 14 Oct 2026

Watermarking Makes Language Models Radioactive

arXiv 2402.14904 ↗First posted 22 Feb 2024cs.CR cs.AI cs.CLcited by 58

On the map: LLM safety and security → Watermarking generated content and models

Abstract (arXiv)

We investigate the radioactivity of text generated by large language models (LLM), i.e. whether it is possible to detect that such synthetic input was used to train a subsequent LLM. Current methods like membership inference or active IP protection either work only in settings where the suspected text is known or do not provide reliable statistical guarantees. We discover that, on the contrary, it is possible to reliably determine if a language model was trained on synthetic data if that data is output by a watermarked LLM. Our new methods, specialized for radioactivity, detects with a provable confidence weak residuals of the watermark signal in the fine-tuned LLM. We link the radioactivity contamination level to the following properties: the watermark robustness, its proportion in the training set, and the fine-tuning process. For instance, if the suspect model is open-weight, we demonstrate that training on watermarked instructions can be detected with high confidence (p-value < 10-5) even when as little as 5% of training text is watermarked.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. Builds on 54 of its 71 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

54 of 71 references.

Closest ancestors · 18

Same direction, newest first

  1. 2024GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trick20 Feb
  2. 2024Adaptive Text Watermark for Large Language Models25 Jan
  3. 2023On the Learnability of Watermarks for Language Models7 Dec
  4. 2023Mark My Words: Analyzing and Evaluating Language Model Watermarks1 Dec
  5. 2023A Semantic Invariant Robust Watermark for Large Language Models10 Oct
  6. 2023Unbiased Watermark for Large Language Models22 Sep
  7. 2023Advancing Beyond Identification: Multi-bit Watermark for Large Language Models1 Aug
  8. 2023Robust Distortion-free Watermarks for Language Models28 Jul
  9. 2023Three Bricks to Consolidate Watermarks for Large Language Models26 Jul
  10. 2023Watermarking Conditional Text Generation for AI Detection: Unveiling Challenges and a Semantic-Aware Watermark Remedy25 Jul
  11. 2023On the Reliability of Watermarks for Large Language Models7 Jun
  12. 2023Undetectable Watermarks for Language Models25 May
6 more
  1. 2023Who Wrote this Code? Watermarking for Code Generation24 May
  2. 2023Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermark17 May
  3. 2023Protecting Language Generation Models via Invisible Watermarking6 Feb
  4. 2023A Watermark for Large Language Models24 Jan
  5. 2022CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks19 Sep
  6. 2021Protecting Intellectual Property of Language Generation APIs with Lexical Watermark5 Dec

Other references · 36

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  10. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  11. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  12. 2019The Curious Case of Neural Text Degenerationcited by 2,916
24 more
  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  2. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  3. 2020Extracting Training Data from Large Language Modelscited by 2,189
  4. 2018Hierarchical Neural Story Generationcited by 1,193
  5. 2021Membership Inference Attacks From First Principlescited by 878
  6. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  7. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  8. 2023Instruction Tuning with GPT-4cited by 686
  9. 2023Textbooks Are All You Needcited by 569
  10. 2019Quantifying the Carbon Emissions of Machine Learningcited by 521
  11. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  12. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 397
  13. 2023Detecting Pretraining Data from Large Language Modelscited by 387
  14. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 379
  15. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 271
  16. 2023Time Travel in LLMs: Tracing Data Contamination in Large Language Modelscited by 177
  17. 2021On the Importance of Difficulty Calibration in Membership Inference Attackscited by 147
  18. 2023Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Taskscited by 103
  19. 2020Imitation Attacks and Defenses for Black-box Machine Translation Systemscited by 81
  20. 2022Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methodscited by 79
  21. 2019Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?cited by 71
  22. 2020Radioactive data: tracing through trainingcited by 67
  23. 2021Membership Inference on Word Embedding and Beyondcited by 35
  24. 2018D'ej`a Vu: an empirical evaluation of the memorization properties of ConvNetscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 26; 2026: 26.

58 papers.

Sort:

By area:

  1. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  2. 2024Membership Inference Attacks Cannot Prove that a Model Was Trained On Your Datacited by 50
  3. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 28
  4. 2025Antidistillation Samplingcited by 25
  5. 2024Ward: Provable RAG Dataset Inference via LLM Watermarkscited by 16
  6. 2025Watermarking Autoregressive Image Generationcited by 16
  7. 2025Scalable Fingerprinting of Large Language Modelscited by 14
  8. 2024Can Watermarked LLMs be Identified by Users via Crafted Prompts?cited by 13
  9. 2024Provably Robust Watermarks for Open-Source Language Modelscited by 13
  10. 2025Seven Security Challenges in Cross-domain Multi-agent LLM Systemscited by 12
  11. 2025Unified attacks to large language model watermarks: spoofing and scrubbing in unauthorized knowledge distillationcited by 11
  12. 2025SeedPrints: Fingerprints Can Even Tell Which Seed Your Large Language Model Was Trained Fromcited by 10
38 more
  1. 2025LLM Fingerprinting via Semantically Conditioned Watermarkscited by 9
  2. 2025STAMP Your Content: Proving Dataset Membership via Watermarked Rephrasingscited by 8
  3. 2025Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoptioncited by 7
  4. 2025Detecting Benchmark Contamination Through Watermarkingcited by 6
  5. 2025Radioactive Watermarks in Diffusion and Autoregressive Image Generative Modelscited by 6
  6. 2026Antidistillation Fingerprintingcited by 6
  7. 2024Building Intelligence Identification System via Large Language Model Watermarking: A Survey and Beyondcited by 5
  8. 2025Obliviate: Efficient Unmemorization for Protecting Intellectual Property in Large Language Modelscited by 5
  9. 2025BitMark: Watermarking Bitwise Autoregressive Image Generative Modelscited by 5
  10. 2025Leave No TRACE: Black-box Detection of Copyrighted Dataset Usage in Large Language Models via Watermarkingcited by 5
  11. 2025Large Language Model Sourcing: A Surveycited by 5
  12. 2026TextSeal: A Localized LLM Watermark for Provenance & Distillation Protectioncited by 5
  13. 2025Data Provenance Auditing of Fine-Tuned Large Language Models with a Text-Preserving Techniquecited by 4
  14. 2025ImageSentinel: Protecting Visual Datasets from Unauthorized Retrieval-Augmented Image Generationcited by 4
  15. 2025PRO: Enabling Precise and Robust Text Watermark for Open-Source LLMscited by 4
  16. 2026Watermarking LLM Agent Trajectoriescited by 4
  17. 2025Watermarking Degrades Alignment in Language Models: Analysis and Mitigationcited by 3
  18. 2026RedAct: Redacting Agent Capability Traces for Procedural Skill Protectioncited by 3
  19. 2025RLCracker: Evaluating the Worst-Case Vulnerability of LLM Watermarks with Adaptive RL Attackscited by 2
  20. 2025How Good is Post-Hoc Watermarking With Language Model Rephrasing?cited by 2
  21. 2026Asking Back: Interaction-Layer Antidistillation Watermarkscited by 2
  22. 2025LLM-Text Watermarking based on Lagrange Interpolationcited by 1
  23. 2025HeavyWater and SimplexWater: Distortion-Free LLM Watermarks for Low-Entropy Next-Token Predictionscited by 1
  24. 2025ActiveMark: on watermarking of visual foundation models via massive activationscited by 1
  25. 2026Lossless Anti-Distillation Samplingcited by 1
  26. 2026Robust LLM Watermarking with Minimal Semantic Distortion for IP Protectioncited by 1
  27. 2026Watermark Forensics for Generative Models: An Information-Theoretic Perspectivecited by 1
  28. 2025Evidencing Unauthorized Training Data from AI Generated Content using Information Isotopescited by 0
  29. 2026TensorCommitments: A Lightweight Verifiable Inference for Language Modelscited by 0
  30. 2026RandMark: On Random Watermarking of Visual Foundation Modelscited by 0
  31. 2026SERUM: Simple, Efficient, Robust, and Unifying Marking for Diffusion-based Image Generationcited by 0
  32. 2026Functional Subspace Watermarking for Large Language Modelscited by 0
  33. 2026FedAttr: Towards Privacy-preserving Client-Level Attribution in Federated LLM Fine-tuningcited by 0
  34. 2026Trustworthy AI: Ensuring Reliability and Accountability from Models to Agentscited by 0
  35. 2026Watermarking Game-Playing Agents in Perfect-Information Extensive-Form Gamescited by 0
  36. 2026LineageMark: Multi-user White-box Watermarking for Contribution Tracing in Model Derivation Chainscited by 0
  37. 2026OVIG: Optimistic Verification of AI Training Integrity via Gradient Signalscited by 0
  38. 2026Watermarking for Proprietary Dataset Protectioncited by 0

50 most cited of 58 All 58 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Watermark Robustness and Radioactivity May Be at Odds in Federated Learning
  2. 2024Learnable Linguistic Watermarks for Tracing Model Extraction Attacks on Large Language Models
  3. 2025Can LLM Watermarks Robustly Prevent Unauthorized Knowledge Distillation?
  4. 2024Black-Box Detection of Language Model Watermarks
  5. 2023Baselines for Identifying Watermarked Large Language Models
  6. 2024Can Watermarking Large Language Models Prevent Copyrighted Text Generation and Hide Training Data?
  7. 2024PostMark: A Robust Blackbox Watermark for Large Language Models
  8. 2025Optimized Couplings for Watermarking Large Language Models