mlsanityData to 7 Oct 2026

Entropy-Aware Token Rejection for Improving Speculative Decoding

arXiv 2512.23765 ↗First posted 29 Dec 2025cs.CL cs.AIcited by 2

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Speculative decoding (SD) accelerates large language model (LLM) inference by using a lightweight draft model to propose tokens and a stronger target model to verify them. However, standard SD is mainly designed for acceleration, and its output quality is typically constrained by the target model. In this work, we propose Entropy-Aware Speculative Decoding (EASD), a lightweight and training-free extension of SD that improves reasoning quality through token-level entropy-guided rejection. EASD detects cases where both draft and target models exhibit high uncertainty while strongly overlapping in their top predictions. In such uncertain-agreement cases, EASD rejects the aligned token and resamples from the target distribution, preventing low-confidence errors from propagating. Experiments on challenging reasoning benchmarks show that EASD consistently improves accuracy over standard SD and reward-guided variants while maintaining comparable inference efficiency. Notably, EASD can surpass the standalone performance of the target model, suggesting that speculative decoding can serve not only as an acceleration method but also as an effective mechanism for improving reasoning quality. The code is available at https://github.com/ECNU-Text-Computing/EASD.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 30 of its 38 references that are in the map. First posted on arXiv 29 Dec 2025.

Builds on

30 of 38 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Reward-Shifted Speculative Sampling Is An Efficient Test-Time Weak-to-Strong Aligner20 Aug
  2. 2025HeteroSpec: Leveraging Contextual Heterogeneity for Efficient Speculative Decoding19 May
  3. 2025SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning10 Apr
  4. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test3 Mar
  5. 2025Reward-Guided Speculative Decoding for Efficient LLM Reasoning31 Jan
  6. 2025Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment31 Jan
  7. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees24 Jun
  8. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  9. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  10. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  11. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 19

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2023Qwen Technical Reportcited by 3,273
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  6. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  7. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  8. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  9. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  10. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  11. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  12. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
7 more
  1. 2025LIMO: Less is More for Reasoningcited by 488
  2. 2024Learning to Decode Collaboratively with Multiple Language Modelscited by 61
  3. 2025Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Timecited by 48
  4. 2024EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Samplingcited by 42
  5. 2024Entropy-Based Decoding for Retrieval-Augmented Large Language Modelscited by 19
  6. 2025Entropy Adaptive Decoding: Dynamic Model Switching for Efficient Inferencecited by 5
  7. 2025Dynamic Collaboration of Multi-Language Models based on Minimal Complete Semantic Unitscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Future Validity is the Missing Statistic: From Impossibility to Phi-Estimation for Grammar-Faithful Speculative Decodingcited by 2
  2. 2026When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitionscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2024AdaEDL: Early Draft Stopping for Speculative Decoding of Large Language Models via an Entropy-based Lower Bound on Token Acceptance Probability
  2. 2025AdaSD: Adaptive Speculative Decoding for Efficient Language Model Inference
  3. 2026From Tokens to Steps: Verification-Aware Speculative Decoding for Efficient Multi-Step Reasoning
  4. 2025Speculative Decoding: Performance or Illusion?
  5. 2025Speculative Verification: Exploiting Information Gain to Refine Speculative Decoding
  6. 2026MARS: Unleashing the Power of Speculative Decoding via Margin-Aware Verification
  7. 2025AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders
  8. 2025Training-Free Loosely Speculative Decoding: Accepting Semantically Correct Drafts Beyond Exact Match