DRelay: Global Draft Context for Prefix-Aware Parallel Speculative Decoding Repair
On the map: Efficient LLM inference and serving → Speculative decoding
Abstract (arXiv)
Parallel drafting reduces the drafting overhead of speculative decoding for large language models (LLMs), but its gains remain limited by the accepted prefix length. Even when the correct token is present in the candidate pool, a single early selection error prevents subsequent predictions from being used. We propose DRelay, which uses global information from the entire draft block to perform prefix-aware selective repair of candidate selections before target-model verification. DRelay bases its decisions on candidate correlations and the selected path: a global reader extracts predictive information across positions for each candidate. While a causal selector combines candidate-level information extracted by the global read with the tokens selected at preceding positions to determine whether the native choice at the current position is consistent with the global evidence and the selected prefix. It then decides whether to retain or replace the token, thereby repairing early errors and extending the accepted prefix. We further jointly train the draft backbone and the selector, combining candidate-support learning with a repair objective, while weighting the repair loss according to each block position's potential contribution to the consecutive accepted prefix. Across eight diverse benchmarks on an H800 GPU, DRelay consistently improves both average acceptance length and end-to-end decoding performance over DFlash, Domino, and DSpark. Under SGLang serving, DRelay improves average end-to-end speedup over DFlash, Domino, and DSpark by 14.7%-16.8%, 8.7%-9.3%, and 8.1%-9.3%, respectively.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 26 of its 29 references that are in the map. First posted on arXiv 1 Oct 2026.
Builds on
26 of 29 references.
Closest ancestors · 17
Same direction, newest first
- 2026Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training7 Sep
- 2026DFlow: Enabling Verifier Information Flow in Block Diffusion Speculative Decoding6 Sep
- 2026Verification-Aware Training for Speculative Decoding31 Aug
- 2026ReTrace: Rejected-Trajectory Conditioning for Speculative Decoding30 Aug
- 2026AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters21 Jul
- 2026DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation6 Jul
- 2026DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding1 Jun
- 2026TAPS: Target-Aware Prefix Tree Selection for Diffusion-Drafted Speculative Decoding30 May
- 2026Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding28 May
- 2026Draft-OPD: On-Policy Distillation for Speculative Draft Models28 May
- 2026D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Drafting12 May
- 2026SpecForge: A Flexible and Efficient Open-Source Training Framework for Speculative Decoding19 Mar
5 more
- 2026DFlash: Block Diffusion for Flash Speculative Decoding5 Feb
- 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test3 Mar
- 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
- 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
- 2022Fast Inference from Transformers via Speculative Decoding30 Nov
Other references · 9
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2025Qwen3 Technical Reportcited by 8,195
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2023Let's Verify Step by Stepcited by 3,598
- 2021Program Synthesis with Large Language Modelscited by 3,509
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
- 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
- 2024The Perfect Blend: Redefining RLHF with Mixture of Judgescited by 43
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026CURE: Local Uncertainty Repair for Block-Parallel Speculative Decoding
- 2026D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models
- 2026DEdit: Iterative Draft Editing for Speculative Decoding
- 2026LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter
- 2025Speculative Verification: Exploiting Information Gain to Refine Speculative Decoding
- 2026DScale: Scaling Block-Diffusion Speculative Decoding with Adaptive Verification
- 2026JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting
- 2024ParallelSpec: Parallel Drafter for Efficient Speculative Decoding