# LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter

arXiv: https://arxiv.org/abs/2609.37029 · HTML: https://mlsanity.com/paper/2609.37029/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving.
- Builds on 31 of its 33 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Placed in

- Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/)

## Builds on (most cited first, 30 of 31)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads](https://mlsanity.com/paper/2401.10774/) (2024)
- [EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty](https://mlsanity.com/paper/2401.15077/) (2024)
- [EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test](https://mlsanity.com/paper/2503.01840/) (2025)
- [Blockwise Parallel Decoding for Deep Autoregressive Models](https://mlsanity.com/paper/1811.03115/) (2018)
- [DFlash: Block Diffusion for Flash Speculative Decoding](https://mlsanity.com/paper/2602.06036/) (2026)
- [DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation](https://mlsanity.com/paper/2607.05147/) (2026)
- [DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding](https://mlsanity.com/paper/2510.02358/) (2025)
- [Recurrent Drafter for Fast Speculative Decoding in Large Language Models](https://mlsanity.com/paper/2403.09919/) (2024)
- [LongCodeBench: Evaluating Coding LLMs at 1M Context Windows](https://mlsanity.com/paper/2505.07897/) (2025)
- [Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding](https://mlsanity.com/paper/2605.29707/) (2026)
- [PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation](https://mlsanity.com/paper/2504.18583/) (2025)
- [Accelerating Speculative Decoding with Block Diffusion Draft Trees](https://mlsanity.com/paper/2604.12989/) (2026)
- [ParallelSpec: Parallel Drafter for Efficient Speculative Decoding](https://mlsanity.com/paper/2410.05589/) (2024)
- [LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification](https://mlsanity.com/paper/2502.17421/) (2025)
- [Structuring The Future: Diffusion LLM Speculative Decoding via Calibrated Draft Graphs](https://mlsanity.com/paper/2509.18085/) (2025)
- [JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting](https://mlsanity.com/paper/2606.18394/) (2026)
- [DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding](https://mlsanity.com/paper/2606.02091/) (2026)
- [Accelerating Production LLMs with Combined Token/Embedding Speculators](https://mlsanity.com/paper/2404.19124/) (2024)
- [xPress: Parallel Refinement for Diffusion Drafters in Speculative Decoding](https://mlsanity.com/paper/2608.02438/) (2026)
- [P-EAGLE: Parallel-Drafting EAGLE with Scalable Training](https://mlsanity.com/paper/2602.01469/) (2026)
- [Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion](https://mlsanity.com/paper/2605.12825/) (2026)
- [AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters](https://mlsanity.com/paper/2607.19223/) (2026)
- [PRESTO: Prefix-Aligned Tree Drafting for Diffusion Speculative Decoding](https://mlsanity.com/paper/2607.22634/) (2026)

## Abstract (arXiv)

Speculative decoding accelerates autoregressive inference by verifying multiple draft tokens in a single target forward pass. However, as the context grows, existing state-of-the-art drafters become increasingly expensive, eroding the very efficiency advantage they are designed to provide. We argue that this scaling is unnecessary. A standalone language model must grow with its prefix because it is solely responsible for every token it produces. A drafter, by contrast, only proposes candidates; the target catches and corrects every error before any token is committed. The drafter's decoding cost can therefore be made entirely independent of the prefix length. We introduce LongSpark, a block-diffusion drafter that achieves this by extracting fixed-size, multiscale views from the target's verification pass, thereby eliminating the need for a growing persistent state. Extensive evaluations demonstrate that LongSpark achieves state-of-the-art end-to-end efficiency across multiple model scales and realistic serving conditions. Notably, it delivers the lowest time-per-output-token on long-context tasks while reducing the drafter's context state by several orders of magnitude.