Beyond Parallel Blindness: Information Floors and Model Gaps in Block Drafting
Not on the map. Why?
Abstract (arXiv)
Block drafters propose several tokens in one forward pass, before earlier target tokens are realised. Their rejection mixes two losses: missing within-block path information and imperfect modelling of observable information. Accepted length cannot distinguish them. We separate the two with an information floor, the minimum expected rejection at a specified conditioning order; rejection above this floor is the model gap. Measurements across four domains and four open-weight targets, plus floor estimates on a frontier API target, yield three findings. First, the all-parallel floor reaches 0.286 at the final slot on Qwen3-4B, limiting even the best proposal to 71% per-slot acceptance. Second, one realised token removes 86--100% of this floor, a locality also recovered by an independent mutual-information analysis. Third, current drafters remain far above their floors: the final-slot model gap accounts for 43--64% of DFlash rejection and 85--92% of DSpark's oracle-conditioned rejection. Guided by this separation of short-range conditioning from proposal quality, we replace a context-independent predecessor correction with a prefix-attention head that reads committed context conditional on the predecessor. With supporting backbone components, the resulting Qwen3-4B drafter improves mean serving accepted length by 3.93% over the released DSpark checkpoint across nine tasks, without increasing the conditioning order. Code and data: https://github.com/tie-pilot-qxw/specfloor; drafter checkpoints: https://huggingface.co/TIE-Pilot/dspark-attnconv-block7-qwen3-4b.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 35 references that are in the map. First posted on arXiv 27 Aug 2026.
Builds on
24 of 35 references.
References · 24
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2021Program Synthesis with Large Language Modelscited by 3,509
- 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
- 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
- 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
- 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 564
- 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 484
- 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 362
- 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 359
- 2024Better & Faster Large Language Models via Multi-token Predictioncited by 344
- 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 318
- 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 318
12 more
- 2022The Parallelism Tradeoff: Limitations of Log-Precision Transformerscited by 191
- 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generationcited by 169
- 2023Speculative Decoding with Big Little Decodercited by 163
- 2023DistillSpec: Improving Speculative Decoding via Knowledge Distillationcited by 160
- 2026DFlash: Block Diffusion for Flash Speculative Decodingcited by 120
- 2024Hydra: Sequentially-Dependent Draft Heads for Medusa Decodingcited by 113
- 2022Formal Language Recognition by Hard Attention Transformers: Perspectives from Circuit Complexitycited by 92
- 2026Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decodingcited by 31
- 2026D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Draftingcited by 16
- 2026xPress: Parallel Refinement for Diffusion Drafters in Speculative Decodingcited by 7
- 2026DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Draftingcited by 2
- 2026From Chains to Trees: Parent-Conditioned Drafting for Semi-Autoregressive Speculative Decodingcited by 2
Built on by
Not cited yet.