# SpecBranch: Speculative Decoding via Hybrid Drafting and Rollback-Aware Branch Parallelism

arXiv: https://arxiv.org/abs/2506.01979 · HTML: https://mlsanity.com/paper/2506.01979/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving.
- First posted on arXiv 16 May 2025.

## Placed in

- Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/)

## Built on by (most cited first, 15 of 15)

- [Speculative Speculative Decoding](https://mlsanity.com/paper/2603.03251/) (2026)
- [ECHO: Elastic Speculative Decoding with Sparse Gating for High-Concurrency Scenarios](https://mlsanity.com/paper/2604.09603/) (2026)
- [Breaking Entropy Bounds: Accelerating RL Training via MTP with Rejection Sampling](https://mlsanity.com/paper/2606.12370/) (2026)
- [Batch Query Processing and Optimization for Agentic Workflows](https://mlsanity.com/paper/2509.02121/) (2025)
- [Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decoding](https://mlsanity.com/paper/2605.20104/) (2026)
- [When Hidden States Drift: Can KV Caches Rescue Long-Range Speculative Decoding?](https://mlsanity.com/paper/2604.26412/) (2026)
- [D-cut: Adaptive Verification Depth Pruning for Batched Speculative Decoding](https://mlsanity.com/paper/2607.14647/) (2026)
- [AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding](https://mlsanity.com/paper/2607.25852/) (2026)
- [FlexDraft: Flexible Speculative Decoding via Attention Tuning and Bonus-Guided Calibration](https://mlsanity.com/paper/2605.20022/) (2026)
- [TailSieve: Partial-Rollout-Guided Tail Routing for LLM Rollouts](https://mlsanity.com/paper/2608.22788/) (2026)
- [FlowHijack: A Dynamics-Aware Backdoor Attack on Flow-Matching Vision-Language-Action Models](https://mlsanity.com/paper/2604.09651/) (2026)
- [FASER: Fine-Grained Phase Management for Speculative Decoding in Dynamic LLM Serving](https://mlsanity.com/paper/2604.20503/) (2026)
- [Dynamic Delayed Tree Expansion For Improved Multi-Path Speculative Decoding](https://mlsanity.com/paper/2602.16994/) (2026)
- [GRAFT: Adaptive DLM-Based Draft Tree Construction with Target-Distilled Edge Scoring](https://mlsanity.com/paper/2608.20375/) (2026)
- [JetSpec++: Foresight Parallel Drafting with Self Refinements](https://mlsanity.com/paper/2610.19792/) (2026)

## Abstract (arXiv)

Recently, speculative decoding (SD) has emerged as a promising technique to accelerate LLM inference by employing a small draft model to propose draft tokens in advance, and validating them in parallel with the large target model. However, the existing SD methods still remain fundamentally constrained by their serialized execution, which causes the mutual waiting bubbles between the draft and target models. To address this challenge, we draw inspiration from branch prediction in modern processors and propose a novel framework SpecBranch to unlock branch parallelism in SD. Specifically, we first take an in-depth analysis of the potential of branch parallelism in SD, and recognize that the key challenge lies in the trade-offs between parallelization and token rollback. Based on the analysis, we strategically introduce parallel speculative branches to preemptively hedge against likely rejections. Meanwhile, to enhance parallelism, we jointly orchestrate adaptive draft lengths with a hybrid combination of the implicit draft model confidence and explicit reusing of target model features. Extensive experiments across various models and benchmarks show that SpecBranch achieves over 1.8× ~ 4.5× speedups against the auto-regressive decoding and reduces rollback tokens by 50% for poorly aligned models, realizing its applicability for real-world deployments.