mlsanityData to 14 Oct 2026

Ceiling-Clipped Acceptance Histograms Indicate Stranded Speed-up in Block-Diffusion Speculative Decoding

arXiv 2608.30427 ↗First posted 31 Aug 2026cs.CL cs.LGcited by 0

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Speculative decoding speeds up generation with an efficient draft model (drafter) that proposes tokens for a target model to verify in one pass, preserving the target's output distribution. High-acceptance block-diffusion drafters such as DFlash and DFlare fill an entire block in one parallel pass. In many cycles, the target accepts the whole block, so the drafter exhausts its trained block horizon before verification fails. We call this unrealized acceptance stranded speed-up. A mean committed length, per prompt or per cycle, hides it, whereas the acceptance histogram exposes it as a spike in the ceiling bin, the fraction of cycles that accept the entire block. We recommend the histogram as a preflight check before spending training compute. Naively widening the block at inference does not recover the speed-up, because once the block outgrows its training size, the drafter's bidirectional attention shifts its distribution even at early positions and erodes front-of-block verification. Instead, we post-train the drafter on a longer block with a short curriculum that emphasizes the newly exposed positions, a method we call DBloom. Expanding the pretrained DFlash and DFlare drafters from block size 16 to 24 across Qwen3-8B and Qwen3-4B targets raises the per-prompt committed length on the high-ceiling benchmarks by a median of +0.8 tokens (up to +1.1). Once continuation fine-tuning precedes expansion, the increase reaches 1.37 tokens. The same expansion also lifts committed length on all seven benchmarks for Gemma-4-12B-IT, a different model family, by a median of +0.41 tokens (Arm A), and the full continuation-then-expand pipeline (Arm B) adds +0.29 to +0.98 tokens over the same B16 drafter. In a prompt-matched comparison against JetSpec, a contemporary tree-based drafter not used in our design, DBloom commits more tokens on every benchmark at tree budgets up to 64 nodes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 13 of its 14 references that are in the map. First posted on arXiv 31 Aug 2026.

Builds on

13 of 14 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting16 Jun
  2. 2026DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding1 Jun
  3. 2026Bastion: Budget-Aware Speculative Decoding with Tree-structured Block Diffusion Drafting28 May
  4. 2026Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding28 May
  5. 2026D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Drafting12 May
  6. 2026Accelerating Speculative Decoding with Block Diffusion Draft Trees14 Apr
  7. 2026DFlash: Block Diffusion for Flash Speculative Decoding5 Feb
  8. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test3 Mar
  9. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  10. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  11. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 2

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2026Teaching Diffusion to Speculate Left-to-Rightcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Accelerating Speculative Diffusions via Block Verification
  2. 2026TAPS: Target-Aware Prefix Tree Selection for Diffusion-Drafted Speculative Decoding
  3. 2026Spec-AUF: Accept-Until-Fail Training under Train-Inference Misalignment for Masked Block Drafters
  4. 2024Block Verification Accelerates Speculative Decoding
  5. 2026D-Loop: Looped Diffusion Drafting for Speculative Decoding
  6. 2026BV Loss: Block Verification-Aware Loss for Block Diffusion Speculative Decoding
  7. 2026Cost-Aware Diffusion Draft Trees for Speculative Decoding
  8. 2026SpecBlock: Block-Iterative Speculative Decoding with Dynamic Tree Drafting