# Entropy-Aware Token Rejection for Improving Speculative Decoding

arXiv: https://arxiv.org/abs/2512.23765 · HTML: https://mlsanity.com/paper/2512.23765/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving.
- Builds on 30 of its 38 references that are in the map.
- First posted on arXiv 29 Dec 2025.

## Placed in

- Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/)

## Builds on (most cited first, 30 of 30)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Solving Quantitative Reasoning Problems with Language Models](https://mlsanity.com/paper/2206.14858/) (2022)
- [AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration](https://mlsanity.com/paper/2306.00978/) (2023)
- [OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems](https://mlsanity.com/paper/2402.14008/) (2024)
- [Scaling Laws for Reward Model Overoptimization](https://mlsanity.com/paper/2210.10760/) (2022)
- [Accelerating Large Language Model Decoding with Speculative Sampling](https://mlsanity.com/paper/2302.01318/) (2023)
- [Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads](https://mlsanity.com/paper/2401.10774/) (2024)
- [EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty](https://mlsanity.com/paper/2401.15077/) (2024)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning](https://mlsanity.com/paper/2506.01939/) (2025)
- [LIMO: Less is More for Reasoning](https://mlsanity.com/paper/2502.03387/) (2025)
- [EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees](https://mlsanity.com/paper/2406.16858/) (2024)
- [EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test](https://mlsanity.com/paper/2503.01840/) (2025)
- [Reward-Guided Speculative Decoding for Efficient LLM Reasoning](https://mlsanity.com/paper/2501.19324/) (2025)
- [Learning to Decode Collaboratively with Multiple Language Models](https://mlsanity.com/paper/2403.03870/) (2024)
- [Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment](https://mlsanity.com/paper/2501.19309/) (2025)
- [SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning](https://mlsanity.com/paper/2504.07891/) (2025)
- [Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Time](https://mlsanity.com/paper/2504.12329/) (2025)
- [EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Sampling](https://mlsanity.com/paper/2403.14541/) (2024)
- [Entropy-Based Decoding for Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2406.17519/) (2024)
- [Entropy Adaptive Decoding: Dynamic Model Switching for Efficient Inference](https://mlsanity.com/paper/2502.06833/) (2025)
- [Reward-Shifted Speculative Sampling Is An Efficient Test-Time Weak-to-Strong Aligner](https://mlsanity.com/paper/2508.15044/) (2025)
- [HeteroSpec: Leveraging Contextual Heterogeneity for Efficient Speculative Decoding](https://mlsanity.com/paper/2505.13254/) (2025)
- [Dynamic Collaboration of Multi-Language Models based on Minimal Complete Semantic Units](https://mlsanity.com/paper/2508.18763/) (2025)

## Built on by (most cited first, 3 of 3)

- [Future Validity is the Missing Statistic: From Impossibility to Phi-Estimation for Grammar-Faithful Speculative Decoding](https://mlsanity.com/paper/2605.07698/) (2026)
- [When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions](https://mlsanity.com/paper/2605.22873/) (2026)
- [Learning What to Imitate: Entropy-Aware Distribution Mixing](https://mlsanity.com/paper/2610.06671/) (2026)

## Abstract (arXiv)

Speculative decoding (SD) accelerates large language model (LLM) inference by using a lightweight draft model to propose tokens and a stronger target model to verify them. However, standard SD is mainly designed for acceleration, and its output quality is typically constrained by the target model. In this work, we propose Entropy-Aware Speculative Decoding (EASD), a lightweight and training-free extension of SD that improves reasoning quality through token-level entropy-guided rejection. EASD detects cases where both draft and target models exhibit high uncertainty while strongly overlapping in their top predictions. In such uncertain-agreement cases, EASD rejects the aligned token and resamples from the target distribution, preventing low-confidence errors from propagating. Experiments on challenging reasoning benchmarks show that EASD consistently improves accuracy over standard SD and reward-guided variants while maintaining comparable inference efficiency. Notably, EASD can surpass the standalone performance of the target model, suggesting that speculative decoding can serve not only as an acceleration method but also as an effective mechanism for improving reasoning quality. The code is available at https://github.com/ECNU-Text-Computing/EASD.