# PEARL: Parallel Speculative Decoding with Adaptive Draft Length arXiv: https://arxiv.org/abs/2408.11850 · HTML: https://mlsanity.com/paper/2408.11850/ ## Key facts - Cited by 85 papers in the map (data through 14 Oct 2026). - Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. - Builds on 19 of its 24 references that are in the map. - First posted on arXiv 13 Aug 2024. ## Placed in - Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/) ## Builds on (most cited first, 19 of 19) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023) - [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015) - [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021) - [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023) - [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021) - [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021) - [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022) - [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015) - [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023) - [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022) - [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024) - [Accelerating Large Language Model Decoding with Speculative Sampling](https://mlsanity.com/paper/2302.01318/) (2023) - [Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads](https://mlsanity.com/paper/2401.10774/) (2024) - [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018) - [Break the Sequential Dependency of LLM Inference Using Lookahead Decoding](https://mlsanity.com/paper/2402.02057/) (2024) - [A Survey on Model Compression for Large Language Models](https://mlsanity.com/paper/2308.07633/) (2023) - [DistillSpec: Improving Speculative Decoding via Knowledge Distillation](https://mlsanity.com/paper/2310.08461/) (2023) - [Natural Language Processing: State of The Art, Current Trends and Challenges](https://mlsanity.com/paper/1708.05148/) (2017) ## Built on by (most cited first, 30 of 85) - [Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment](https://mlsanity.com/paper/2501.19309/) (2025) - [SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoning](https://mlsanity.com/paper/2505.11484/) (2025) - [A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models](https://mlsanity.com/paper/2508.08712/) (2025) - [Speculative Decoding and Beyond: An In-Depth Survey of Techniques](https://mlsanity.com/paper/2502.19732/) (2025) - [ParallelSpec: Parallel Drafter for Efficient Speculative Decoding](https://mlsanity.com/paper/2410.05589/) (2024) - [LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification](https://mlsanity.com/paper/2502.17421/) (2025) - [Dynamic Depth Decoding: Faster Speculative Decoding for LLMs](https://mlsanity.com/paper/2409.00142/) (2024) - [AdaEAGLE: Optimizing Speculative Decoding via Explicit Modeling of Adaptive Draft Structures](https://mlsanity.com/paper/2412.18910/) (2024) - [Draft Model Knows When to Stop: Self-Verification Speculative Decoding for Long-Form Generation](https://mlsanity.com/paper/2411.18462/) (2024) - [TETRIS: Optimal Draft Token Selection for Batch Speculative Decoding](https://mlsanity.com/paper/2502.15197/) (2025) - [Speculative Speculative Decoding](https://mlsanity.com/paper/2603.03251/) (2026) - [DREAM: Drafting with Refined Target Features and Entropy-Adaptive Cross-Attention Fusion for Multimodal Speculative Decoding](https://mlsanity.com/paper/2505.19201/) (2025) - [LogitSpec: Accelerating Retrieval-based Speculative Decoding via Next Next Token Speculation](https://mlsanity.com/paper/2507.01449/) (2025) - [Utility-Driven Speculative Decoding for Mixture-of-Experts](https://mlsanity.com/paper/2506.20675/) (2025) - [TALON: Confidence-Aware Speculative Decoding with Adaptive Token Trees](https://mlsanity.com/paper/2601.07353/) (2026) - [Fuzzy Speculative Decoding for a Tunable Accuracy-Runtime Tradeoff](https://mlsanity.com/paper/2502.20704/) (2025) - [Dynamic Speculative Agent Planning](https://mlsanity.com/paper/2509.01920/) (2025) - [The Art of Scaling Test-Time Compute for Large Language Models](https://mlsanity.com/paper/2512.02008/) (2025) - [ECHO: Elastic Speculative Decoding with Sparse Gating for High-Concurrency Scenarios](https://mlsanity.com/paper/2604.09603/) (2026) - [SpecEdge: Scalable Edge-Assisted Serving Framework for Interactive LLMs](https://mlsanity.com/paper/2505.17052/) (2025) - [SwiftSpec: Ultra-Low Latency LLM Decoding by Scaling Asynchronous Speculative Decoding](https://mlsanity.com/paper/2506.11309/) (2025) - [TAPS: Target-Aware Prefix Tree Selection for Diffusion-Drafted Speculative Decoding](https://mlsanity.com/paper/2606.00487/) (2026) - [PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decoding](https://mlsanity.com/paper/2605.08632/) (2026) - [Exploiting Edited Large Language Models as General Scientific Optimizers](https://mlsanity.com/paper/2503.09620/) (2025) - [CSV-Decode: Certifiable Sub-Vocabulary Decoding for Efficient Large Language Model Inference](https://mlsanity.com/paper/2511.21702/) (2025) - [Token-Driven GammaTune: Adaptive Calibration for Enhanced Speculative Decoding](https://mlsanity.com/paper/2504.00030/) (2025) - [Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decoding](https://mlsanity.com/paper/2605.20104/) (2026) - [DSD: A Distributed Speculative Decoding Solution for Edge-Cloud Agile Large Model Serving](https://mlsanity.com/paper/2511.21669/) (2025) - [See the Forest for the Trees: Loosely Speculative Decoding via Visual-Semantic Guidance for Efficient Inference of Video LLMs](https://mlsanity.com/paper/2604.05650/) (2026) - [When Hidden States Drift: Can KV Caches Rescue Long-Range Speculative Decoding?](https://mlsanity.com/paper/2604.26412/) (2026) ## Abstract (arXiv) Speculative decoding (SD), where an extra draft model is employed to provide multiple draft tokens first, and then the original target model verifies these tokens in parallel, has shown great power for LLM inference acceleration. However, existing SD methods suffer from the mutual waiting problem, i.e., the target model gets stuck when the draft model is guessing tokens, and vice versa. This problem is directly incurred by the asynchronous execution of the draft model and the target model and is exacerbated due to the fixed draft length in speculative decoding. To address these challenges, we propose a conceptually simple, flexible, and general framework to boost speculative decoding, namely Parallel spEculative decoding with Adaptive dRaft Length (PEARL). Specifically, PEARL proposes pre-verify to verify the first draft token in advance during the drafting phase, and post-verify to generate more draft tokens during the verification phase. PEARL parallels the drafting phase and the verification phase via applying the two strategies, and achieves adaptive draft length for different scenarios, which effectively alleviates the mutual waiting problem. Experiments on various text generation benchmarks demonstrate the effectiveness of our PEARL, leading to a superior speed up performance up to 4.43× and 1.50×, compared to auto-regressive decoding and vanilla speculative decoding, respectively. Our code is available at https://github.com/smart-lty/ParallelSpeculativeDecoding.