mlsanityData to 14 Oct 2026

Vegas: Self-Speculative Decoding with Verification-Guided Sparse Attention

arXiv 2602.07223 ↗First posted 6 Feb 2026cs.LGcited by 7

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Long-context large language model (LLM) inference has become the norm for today's AI applications. However, it is severely bottlenecked by the increasing memory demands of its KV cache. Previous works have shown that self-speculative decoding with sparse attention, where tokens are drafted using a subset of the KV cache and verified in parallel against the full KV cache, speeds up inference in a lossless manner. However, they rely on a standalone KV selection algorithm to select the KV entries used for drafting and overlook the fact that the criticality of each KV entry is inherently computed during verification. In this paper, we propose Vegas, a self-speculative decoding method with verification-guided sparse attention. Vegas identifies critical KV cache entries as a byproduct of verification and computes attention only over these entries when drafting subsequent tokens. This not only improves the draft token acceptance rate but also incurs low KV selection overhead, thereby improving decoding throughput. Vegas achieves a 1.25×-2.81× speedup in decoding throughput over default vLLM and a 1.15×-1.29× speedup over state-of-the-art sparse attention-based self-speculative decoding methods. Our code is available at https://github.com/platformxlab/vegas.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 44 of its 53 references that are in the map. First posted on arXiv 6 Feb 2026.

Builds on

44 of 53 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding1 Dec
  2. 2025SpecExtend: A Drop-in Enhancement for Speculative Decoding of Long Sequences27 May
  3. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test3 Mar
  4. 2025RAPID: Long-Context Inference with Retrieval-Augmented Speculative Decoding27 Feb
  5. 2025QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache5 Feb
  6. 2024SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration9 Oct
  7. 2024MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding20 Aug
  8. 2024TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding18 Apr
  9. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decoding3 Feb
  10. 2023DistillSpec: Improving Speculative Decoding via Knowledge Distillation12 Oct
  11. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  12. 2022Fast Inference from Transformers via Speculative Decoding30 Nov
1 more
  1. 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation30 Mar

Other references · 31

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  6. 2020Longformer: The Long-Document Transformercited by 3,413
  7. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  8. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  9. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  10. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 519
  11. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  12. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 474
19 more
  1. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 407
  2. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 379
  3. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 307
  4. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headscited by 288
  5. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 258
  6. 2024InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memorycited by 197
  7. 2024Transformers are Multi-State RNNscited by 120
  8. 2025A Comprehensive Survey on Long Context Language Modelingcited by 117
  9. 2025CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratingscited by 115
  10. 2024MagicPIG: LSH Sampling for Efficient LLM Generationcited by 111
  11. 2024PQCache: Product Quantization-based KVCache for Long Context LLM Inferencecited by 95
  12. 2024ClusterKV: Manipulating LLM KV Cache in Semantic Space for Recallable Compressioncited by 65
  13. 2024SCBench: A KV Cache-Centric Analysis of Long-Context Methodscited by 60
  14. 2025Twilight: Adaptive Attention Sparsity with Hierarchical Top-p Pruningcited by 49
  15. 2024SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attentioncited by 39
  16. 2024TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selectioncited by 39
  17. 2025SpecVLM: Enhancing Speculative Decoding of Video LLMs via Verifier-Guided Token Pruningcited by 20
  18. 2024How Sparse Attention Approximates Exact Attention? Your Attention is Naturally nC-Sparsecited by 17
  19. 2025RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inferencecited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026A Sparse Glimpse of the Whole: Train-Free Self-Speculative Decodingcited by 1
  2. 2026ASPIRE: Asynchronous Batched Self-Speculative Decoding for Long-Context LLM Inferencecited by 1
  3. 2026Spectral Origins of the Self-Correction Blind Spot in Autoregressive Generationcited by 0
  4. 2026Self-Speculation for Faster Reasoning Modelscited by 0
  5. 2026Where Activation Sparsity and KV-Cache Sparsity Cross in LLM Decodingcited by 0
  6. 2026S3R: Fast and Faithful RL Rollouts via Self-Sparse Speculative Decodingcited by 0
  7. 2026FOLD: Position-Free Far Memory with Local Order for Long-Context Speculative Decodingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding
  2. 2025SPIRe: Boosting LLM Inference Throughput with Speculative Decoding
  3. 2025SpecAttn: Speculating Sparse Attention
  4. 2026KnapSpec: Self-Speculative Decoding via Adaptive Layer Selection as a Knapsack Problem
  5. 2023Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding
  6. 2026OasisKV: Scaling In-Decode KV Cache Beyond HBM with Lookahead Sparse Prefetching
  7. 2026Lynx: Progressive Speculative Quantization for accelerating KV Transfer in Long-Context Inference
  8. 2026BudgetDraft: Acceptance-Aware Multi-View Training for Sparse-KV Speculative Decoding