mlsanityData to 7 Oct 2026

From Quarter to All: Accelerating Speculative LLM Decoding via Floating-Point Exponent Remapping and Parameter Sharing

arXiv 2510.18525 ↗First posted 21 Oct 2025cs.ARcited by 1

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Large language models achieve impressive performance across diverse tasks but exhibit high inference latency due to their large parameter sizes. While quantization reduces model size, it often leads to performance degradation compared to the full model. Speculative decoding remains lossless but typically incurs extra overheads. We propose SPEQ, an algorithm-hardware co-designed speculative decoding method that uses part of the full-model weight bits to form a quantized draft model, thereby eliminating additional training or storage overhead. A reconfigurable processing element array enables efficient execution of both the draft and verification passes. Experimental results across 15 LLMs and tasks demonstrate that SPEQ achieves speedups of 2.07x, 1.53x, and 1.45x compared over FP16, Olive, and Tender, respectively.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 19 of its 20 references that are in the map. First posted on arXiv 21 Oct 2025.

Builds on

19 of 20 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration9 Oct
  2. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  3. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  4. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification16 May
  5. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 14

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2024Qwen2.5 Technical Reportcited by 4,713
  7. 2016Pointer Sentinel Mixture Modelscited by 2,992
  8. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  9. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  10. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  11. 2023OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantizationcited by 83
  12. 2024Exploiting LLM Quantizationcited by 61
2 more
  1. 2025Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoningcited by 36
  2. 2024Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantizationcited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recoverycited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023Accelerating LLM Inference with Staged Speculative Decoding
  2. 2025Yggdrasil: Bridging Dynamic Speculation and Static Runtime for Latency-Optimal Tree-Based LLM Decoding
  3. 2025Scaling LLM Speculative Decoding: Non-Autoregressive Forecasting in Large-Batch Scenarios
  4. 2026BitNest: Bit-Nested Speculative Decoding for Memory-Efficient LLM Inference Acceleration
  5. 2024SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices
  6. 2024Decoding Speculative Decoding
  7. 2024Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism
  8. 2025Speculative Decoding Meets Quantization: Compatibility Evaluation and Hierarchical Framework Design