mlsanityData to 14 Oct 2026

Watermarkable Multi-Draft Speculative Sampling via Poisson Processes

arXiv 2609.21858 ↗First posted 18 Sep 2026cs.CR cs.LGcited by 0

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Large language models (LLMs) have achieved state-of-the-art performance across a wide range of tasks, motivating two important aspects of deployment: inference efficiency and output provenance, which can be tackled by speculative sampling and watermarking, respectively. However, recent works have shown that combining these two goals is highly nontrivial and can be potentially impossible. In this work, we develop a novel multi-draft speculative sampling algorithm based on Poisson processes that improves the frontier of this fundamental trade-off. The proposed algorithm has strong sampling efficiency on its own and, more interestingly, is naturally watermarkable: we can embed an unbiased watermark without degrading speculative acceptance. Moreover, our algorithm is based on an exact list-coupling-without-communication scheme, which yields a drafter invariance property that benefits both sampling and watermarking. It is the first multi-draft, drafter-invariant speculative sampling scheme that maintains both watermark strength and sampling efficiency, and we experimentally verify its strong performance in both aspects.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 43 of its 62 references that are in the map. First posted on arXiv 18 Sep 2026.

Builds on

43 of 62 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026Accelerating Diffusion Sampling via Speculative Draft Trees15 Sep
  2. 2026Improving the Trade-off Between Watermark Strength and Speculative Sampling Efficiency for Language Models1 Feb
  3. 2025List-Level Distribution Coupling with Applications to Speculative Decoding and Lossy Compression5 Jun
  4. 2025Speculative Sampling via Exponential Races21 Apr
  5. 2025Accelerated Diffusion Models via Speculative Sampling9 Jan
  6. 2024Inevitable Trade-off between Watermark Strength and Speculative Sampling Efficiency for Language Models27 Oct
  7. 2024Multi-Draft Speculative Sampling: Canonical Decomposition and Theoretical Limits23 Oct
  8. 2023SpecTr: Fast Speculative Decoding via Optimal Transport23 Oct
  9. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification16 May
  10. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  11. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  4. 2018Hierarchical Neural Story Generationcited by 1,193
  5. 2025OpenAI GPT-5 System Cardcited by 989
  6. 2023A Watermark for Large Language Modelscited by 725
  7. 2019ELI5: Long Form Question Answeringcited by 575
  8. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 323
  9. 2023Robust Distortion-free Watermarks for Language Modelscited by 320
  10. 2023Provable Robust Watermarking for AI-Generated Textcited by 299
  11. 2023Undetectable Watermarks for Language Modelscited by 226
  12. 2014A* Samplingcited by 151
20 more
  1. 2017Controlling Linguistic Style Aspects in Neural Language Generationcited by 149
  2. 2023Unbiased Watermark for Large Language Modelscited by 111
  3. 2023A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directionscited by 105
  4. 2024A Statistical Framework of Watermarks for Large Language Models: Pivot, Detection Efficiency and Optimal Rulescited by 57
  5. 2023A Resilient and Accessible Distribution-Preserving Watermark for Large Language Modelscited by 56
  6. 2024WaterMax: breaking the LLM watermark detectability-robustness-quality trade-offcited by 54
  7. 2024Provably Robust Multi-bit Watermarking for AI-generated Textcited by 47
  8. 2021Algorithms for the Communication of Samplescited by 34
  9. 2024Permute-and-Flip: An optimally stable and watermarkable decoder for LLMscited by 22
  10. 2024Multi-Bit Distortion-Free Watermarking for Large Language Modelscited by 11
  11. 2025Optimized Couplings for Watermarking Large Language Modelscited by 9
  12. 2024Universal Exact Compression of Differentially Private Mechanismscited by 8
  13. 2024Coupling without Communication and Drafter-Invariant Speculative Decodingcited by 6
  14. 2024Efficiently Identifying Watermarked Segments in Mixed-Source Textscited by 6
  15. 2025WaterMod: Modular Token-Rank Partitioning for Probability-Balanced LLM Watermarkingcited by 6
  16. 2025On the Empirical Power of Goodness-of-Fit Tests in Watermark Detectioncited by 5
  17. 2026Dual-Representation Image Compression at Ultra-Low Bitrates via Explicit Semantics and Implicit Texturescited by 5
  18. 2024Accelerating Relative Entropy Coding with Space Partitioningcited by 4
  19. 2026Scalable Differentially Private Data Compression via Diffusion and Stochastic Codescited by 3
  20. 2026Refined Detection for Gumbel Watermarkingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Watermarking Discrete Diffusion Language Models
  2. 2025DMark: Order-Agnostic Watermarking for Diffusion Large Language Models
  3. 2026WorldCup Sampling for Multi-bit LLM Watermarking
  4. 2026Linear Ensembles Wash Away Watermarks: On the Fragility of Distributional Perturbations in LLMs
  5. 2025Optimal Detection for Language Watermarks with Pseudorandom Collision
  6. 2025MorphMark: Flexible Adaptive Watermarking for Large Language Models
  7. 2023Three Bricks to Consolidate Watermarks for Large Language Models
  8. 2026dgMARK: Decoding-Guided Watermarking for Diffusion Language Models