mlsanityData to 7 Oct 2026

Quasar: Quantized Self-Speculative Acceleration for Rapid Inference via Memory-Efficient Verification

arXiv 2603.01399 ↗First posted 2 Mar 2026cs.DC cs.LGcited by 1

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Speculative Decoding (SD) has emerged as a premier technique for accelerating Large Language Model (LLM) inference by decoupling token generation into rapid drafting and parallel verification. While recent advancements in self-speculation and lookahead decoding have successfully minimized drafting overhead, they have shifted the primary performance bottleneck to the verification phase. Since verification requires a full forward pass of the target model, it remains strictly memory-bandwidth bound, fundamentally limiting the maximum achievable speedup.In this paper, we introduce Quasar (Quantized Self-speculative Acceleration for Rapid Inference), a novel, training-free framework designed to overcome this "memory wall" by employing low-bit quantization specifically for the verification stage. Our empirical analysis reveals that while aggressive structural pruning significantly degrades verification accuracy, quantization-based verification preserves the logit distribution with high fidelity while effectively halving memory traffic. Extensive experiments on state-of-the-art models (e.g., OpenPangu and Qwen3) demonstrate that Quasar maintains a speculative acceptance length comparable to full-precision methods while achieving a 1.28× improvement in end-to-end throughput. Being orthogonal to existing drafting strategies, Quasar offers a generic and efficient pathway to accelerate the verification leg of speculative execution. Code is available at https://github.com/Tom-HG/Quasar.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 29 of its 35 references that are in the map. First posted on arXiv 2 Mar 2026.

Builds on

29 of 35 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test3 Mar
  2. 2024PLD+: Accelerating LLM inference by leveraging Language Model Artifacts2 Dec
  3. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees24 Jun
  4. 2024SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices4 Jun
  5. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decoding3 Feb
  6. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  7. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  8. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding15 Jan
  9. 2023REST: Retrieval-Based Speculative Decoding14 Nov
  10. 2023SpecTr: Fast Speculative Decoding via Optimal Transport23 Oct
  11. 2023Accelerating Transformer Inference for Translation via Parallel Decoding17 May
  12. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification16 May
2 more
  1. 2023Speculative Decoding with Big Little Decoder15 Feb
  2. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 15

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  8. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  9. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  10. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  11. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  12. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 318
3 more
  1. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 302
  2. 2025Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognitioncited by 61
  3. 2024Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recyclingcited by 48

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recoverycited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Speculative Decoding Meets Quantization: Compatibility Evaluation and Hierarchical Framework Design
  2. 2025QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache
  3. 2024QSpec: Speculative Decoding with Complementary Quantization Schemes
  4. 2025ML-SpecQD: Multi-Level Speculative Decoding with Quantized Drafts
  5. 2024Block Verification Accelerates Speculative Decoding
  6. 2025HiSpec: Hierarchical Speculative Decoding for LLMs
  7. 2023Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding
  8. 2026CATS: Cascaded Adaptive Tree Speculation for Memory-Limited LLM Inference Acceleration