# Minimax bounds for watermarked and masked recursive discrete distribution estimation

arXiv: https://arxiv.org/abs/2608.31091 · HTML: https://mlsanity.com/paper/2608.31091/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 30 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Builds on (most cited first, 24 of 24)

- [Self-Consuming Generative Models Go MAD](https://mlsanity.com/paper/2307.01850/) (2023)
- [Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data](https://mlsanity.com/paper/2404.01413/) (2024)
- [On the Stability of Iterative Retraining of Generative Models on their own Data](https://mlsanity.com/paper/2310.00429/) (2023)
- [Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models](https://mlsanity.com/paper/2311.04378/) (2023)
- [Model Collapse Demystified: The Case of Regression](https://mlsanity.com/paper/2402.07712/) (2024)
- [How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse](https://mlsanity.com/paper/2404.05090/) (2024)
- [Strong Model Collapse](https://mlsanity.com/paper/2410.04840/) (2024)
- [Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World](https://mlsanity.com/paper/2410.16713/) (2024)
- [No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choices](https://mlsanity.com/paper/2402.16187/) (2024)
- [SynthID-Image: Image watermarking at internet scale](https://mlsanity.com/paper/2510.09263/) (2025)
- [Towards Theoretical Understandings of Self-Consuming Generative Models](https://mlsanity.com/paper/2402.11778/) (2024)
- [Rate of Model Collapse in Recursive Training](https://mlsanity.com/paper/2412.17646/) (2024)
- [An Overview of Large Language Models for Statisticians](https://mlsanity.com/paper/2502.17814/) (2025)
- [Heat Death of Generative Models in Closed-Loop Learning](https://mlsanity.com/paper/2404.02325/) (2024)
- [A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspective](https://mlsanity.com/paper/2509.16499/) (2025)
- [A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loops](https://mlsanity.com/paper/2502.18865/) (2025)
- [When Models Don't Collapse: On the Consistency of Iterative MLE](https://mlsanity.com/paper/2505.19046/) (2025)
- [Escaping Collapse: The Strength of Weak Data for Large Language Model Training](https://mlsanity.com/paper/2502.08924/) (2025)
- [Why Less is More (Sometimes): A Theory of Data Curation](https://mlsanity.com/paper/2511.03492/) (2025)
- [What happens when generative AI models train recursively on each others' outputs?](https://mlsanity.com/paper/2505.21677/) (2025)
- [Learning from positive and unlabeled examples -Finite size sample bounds](https://mlsanity.com/paper/2507.07354/) (2025)
- [Language Generation with Replay: A Learning-Theoretic View of Model Collapse](https://mlsanity.com/paper/2603.11784/) (2026)
- [A Probabilistic Perspective on Model Collapse](https://mlsanity.com/paper/2505.13947/) (2025)
- [Learning from Synthetic Data: Limitations of ERM](https://mlsanity.com/paper/2601.15468/) (2026)

## Abstract (arXiv)

Watermarking has been proposed as a way to identify synthetic samples in estimation settings where no metadata is available to distinguish them from real samples, but its precise effects remain unexplored. In the absence of a distinguishing mechanism, it has been shown that adding synthetic samples significantly reduces the marginal efficacy of new real samples. In this work, we study the minimax loss of such recursive discrete distribution estimation in the presence of watermarks in contrast to the unassisted and oracle-assisted losses. When the fraction of real samples vanishes asymptotically, we provide a lower bound that shows that it is impossible to improve performance by adding watermarks unless the false negative rate of detection also vanishes. Additionally, we show that in most regimes, the worst-case losses of a sequence of simple deterministic estimators match the corresponding lower bounds up to constants. Finally, we propose masking, a randomization procedure that narrows the gap in the remaining regimes to a Jensen gap. We conjecture that a tighter lower bound argument can close this gap.