mlsanityData to 14 Oct 2026

SpecPV: Improving Self-Speculative Decoding for Long-Context Generation via Partial Verification

arXiv 2512.02337 ↗First posted 2 Dec 2025cs.LGcited by 7

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Growing demands from tasks like code generation, deep reasoning, and long-document understanding have made long-context generation a crucial capability for large language models (LLMs). Speculative decoding is one of the most direct and effective approaches for accelerating generation. It follows a draft-verify paradigm, where a lightweight draft model proposes several candidate tokens and the target model verifies them. However, we find that as the context length grows, verification becomes the dominant bottleneck. To further accelerate speculative decoding in long-context generation, we introduce SpecPV, a self-speculative decoding approach that performs fast verification using partial key-value states (KV) and periodically applies full verification to eliminate accumulated errors. We validate SpecPV across multiple long-context benchmarks and models, including LLaMA-3.1-8B-Instruct and Qwen3-series. Experimental results show that SpecPV achieves up to 6x decoding speedup over standard autoregressive decoding with minor degradation.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 33 of its 43 references that are in the map. First posted on arXiv 2 Dec 2025.

Builds on

33 of 43 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test3 Mar
  2. 2024SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration9 Oct
  3. 2024MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding20 Aug
  4. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees24 Jun
  5. 2024TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding18 Apr
  6. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  7. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  8. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification16 May
  9. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 24

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  5. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  6. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  7. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  8. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
  9. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  10. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 726
  11. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  12. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
12 more
  1. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  2. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 532
  3. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  4. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 379
  5. 2021QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarizationcited by 365
  6. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 360
  7. 2021Efficient Attentions for Long Document Summarizationcited by 345
  8. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 307
  9. 2025Kimi Linear: An Expressive, Efficient Attention Architecturecited by 196
  10. 2025DeepSeek-OCR: Contexts Optical Compressioncited by 183
  11. 2025XAttention: Block Sparse Attention with Antidiagonal Scoringcited by 137
  12. 2025LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attentioncited by 41

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026HeiSD: Hybrid Speculative Decoding for Embodied Vision-Language-Action Models with Kinematic Awarenesscited by 6
  2. 2026Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decodingcited by 6
  3. 2026Test-Time Speculationcited by 3
  4. 2026TailSieve: Partial-Rollout-Guided Tail Routing for LLM Rolloutscited by 2
  5. 2026SENSE: Semantic Embedding Navigation with Soft-gated Evaluation for Retrieval-based Speculative Decodingcited by 1
  6. 2026Approximate Speculative Decodingcited by 1
  7. 2026From Positionwise Confidence to Prefix Scheduling: Verifier Skipping in Speculative Decodingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Draft Model Knows When to Stop: Self-Verification Speculative Decoding for Long-Form Generation
  2. 2026From Tokens to Steps: Verification-Aware Speculative Decoding for Efficient Multi-Step Reasoning
  3. 2025CLaSp: In-Context Layer Skip for Self-Speculative Decoding
  4. 2025Speculative Verification: Exploiting Information Gain to Refine Speculative Decoding
  5. 2026SpecTr-GBV: Multi-Draft Block Verification Accelerating Speculative Decoding
  6. 2026SpecForge: A Flexible and Efficient Open-Source Training Framework for Speculative Decoding
  7. 2025OWL: Overcoming Window Length-Dependence in Speculative Decoding for Long-Context Inputs
  8. 2025Speculative Decoding: Performance or Illusion?