mlsanityData to 14 Oct 2026

Minimax bounds for watermarked and masked recursive discrete distribution estimation

arXiv 2608.31091 ↗First posted 31 Aug 2026cs.IT cs.LG math.ITcited by 0

Not on the map. Why?

Abstract (arXiv)

Watermarking has been proposed as a way to identify synthetic samples in estimation settings where no metadata is available to distinguish them from real samples, but its precise effects remain unexplored. In the absence of a distinguishing mechanism, it has been shown that adding synthetic samples significantly reduces the marginal efficacy of new real samples. In this work, we study the minimax loss of such recursive discrete distribution estimation in the presence of watermarks in contrast to the unassisted and oracle-assisted losses. When the fraction of real samples vanishes asymptotically, we provide a lower bound that shows that it is impossible to improve performance by adding watermarks unless the false negative rate of detection also vanishes. Additionally, we show that in most regimes, the worst-case losses of a sequence of simple deterministic estimators match the corresponding lower bounds up to constants. Finally, we propose masking, a randomization procedure that narrows the gap in the remaining regimes to a Jensen gap. We conjecture that a tighter lower bound argument can close this gap.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 30 references that are in the map. First posted on arXiv 31 Aug 2026.

Builds on

24 of 30 references.

References · 24

  1. 2023Self-Consuming Generative Models Go MADcited by 250
  2. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Datacited by 151
  3. 2023On the Stability of Iterative Retraining of Generative Models on their own Datacited by 100
  4. 2023Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Modelscited by 87
  5. 2024Model Collapse Demystified: The Case of Regressioncited by 82
  6. 2024How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapsecited by 79
  7. 2024Strong Model Collapsecited by 65
  8. 2024Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating Worldcited by 52
  9. 2024No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choicescited by 50
  10. 2025SynthID-Image: Image watermarking at internet scalecited by 31
  11. 2024Towards Theoretical Understandings of Self-Consuming Generative Modelscited by 29
  12. 2024Rate of Model Collapse in Recursive Trainingcited by 25
12 more
  1. 2025An Overview of Large Language Models for Statisticianscited by 24
  2. 2024Heat Death of Generative Models in Closed-Loop Learningcited by 22
  3. 2025A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspectivecited by 18
  4. 2025A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loopscited by 16
  5. 2025When Models Don't Collapse: On the Consistency of Iterative MLEcited by 13
  6. 2025Escaping Collapse: The Strength of Weak Data for Large Language Model Trainingcited by 11
  7. 2025Why Less is More (Sometimes): A Theory of Data Curationcited by 8
  8. 2025What happens when generative AI models train recursively on each others' outputs?cited by 6
  9. 2025Learning from positive and unlabeled examples -Finite size sample boundscited by 6
  10. 2026Language Generation with Replay: A Learning-Theoretic View of Model Collapsecited by 5
  11. 2025A Probabilistic Perspective on Model Collapsecited by 4
  12. 2026Learning from Synthetic Data: Limitations of ERMcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Sample Complexities of Estimating Gumbel--Max Watermark Proportions with and without Reduction to Pivotal Statistics
  2. 2023Towards Optimal Statistical Watermarking
  3. 2026DRIFT: Removing Diffusion Watermarks by Deflecting the Generative Trajectory
  4. 2026Linear Ensembles Wash Away Watermarks: On the Fragility of Distributional Perturbations in LLMs
  5. 2025Watermarking Discrete Diffusion Language Models
  6. 2026Improving the Trade-off Between Watermark Strength and Speculative Sampling Efficiency for Language Models
  7. 2025Optimal Detection for Language Watermarks with Pseudorandom Collision
  8. 2025The Coding Limits of Robust Watermarking for Generative Models