# LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding

arXiv: https://arxiv.org/abs/2602.23881 · HTML: https://mlsanity.com/paper/2602.23881/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving.
- Builds on 20 of its 26 references that are in the map.
- First posted on arXiv 27 Feb 2026.

## Placed in

- Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/)

## Builds on (most cited first, 20 of 20)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Accelerating Large Language Model Decoding with Speculative Sampling](https://mlsanity.com/paper/2302.01318/) (2023)
- [Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads](https://mlsanity.com/paper/2401.10774/) (2024)
- [EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test](https://mlsanity.com/paper/2503.01840/) (2025)
- [DistillSpec: Improving Speculative Decoding via Knowledge Distillation](https://mlsanity.com/paper/2310.08461/) (2023)
- [Infinity Instruct: Scaling Instruction Selection and Synthesis to Enhance Language Models](https://mlsanity.com/paper/2506.11116/) (2025)
- [A Theoretical Perspective for Speculative Decoding Algorithm](https://mlsanity.com/paper/2411.00841/) (2024)
- [Tailoring Language Generation Models under Total Variation Distance](https://mlsanity.com/paper/2302.13344/) (2023)
- [SpecDiff-2: Scaling Diffusion Drafter Alignment For Faster Speculative Decoding](https://mlsanity.com/paper/2511.00606/) (2025)
- [Speculative Decoding: Performance or Illusion?](https://mlsanity.com/paper/2601.11580/) (2025)
- [FastMTP: Accelerating LLM Inference with Enhanced Multi-Token Prediction](https://mlsanity.com/paper/2509.18362/) (2025)
- [Accelerating Production LLMs with Combined Token/Embedding Speculators](https://mlsanity.com/paper/2404.19124/) (2024)
- [AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders](https://mlsanity.com/paper/2510.19779/) (2025)

## Built on by (most cited first, 18 of 18)

- [Kimi K3: Open Frontier Intelligence](https://mlsanity.com/paper/2607.24653/) (2026)
- [Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding](https://mlsanity.com/paper/2607.05722/) (2026)
- [D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Drafting](https://mlsanity.com/paper/2605.18810/) (2026)
- [Breaking Entropy Bounds: Accelerating RL Training via MTP with Rejection Sampling](https://mlsanity.com/paper/2606.12370/) (2026)
- [AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding](https://mlsanity.com/paper/2607.25852/) (2026)
- [Intern-S2-Preview: Scientific Agentic Foundation Model](https://mlsanity.com/paper/2608.13505/) (2026)
- [Teaching Diffusion to Speculate Left-to-Right](https://mlsanity.com/paper/2606.11552/) (2026)
- [Spec-AUF: Accept-Until-Fail Training under Train-Inference Misalignment for Masked Block Drafters](https://mlsanity.com/paper/2607.01893/) (2026)
- [Trees from Marginals: Autoregressive drafting with factorized priors](https://mlsanity.com/paper/2607.06763/) (2026)
- [A Practical Investigation of Training-free Relaxed Speculative Decoding](https://mlsanity.com/paper/2607.08690/) (2026)
- [The Limits of Speculation: Bounding Speculative Decoding in Mixture-of-Experts](https://mlsanity.com/paper/2609.22156/) (2026)
- [H-Spec: Parallel Speculative Decoding Without a Drafter-Side KV Cache](https://mlsanity.com/paper/2609.24197/) (2026)
- [SlimSpec: Low-Rank Draft LM-Head for Accelerated Speculative Decoding](https://mlsanity.com/paper/2605.10453/) (2026)
- [Jina-OCR-v1: Efficient Document Parsing with Speculative Decoding and Dense Verifiable Rewards](https://mlsanity.com/paper/2609.03181/) (2026)
- [Acceptance-Aware Draft Model Training for Speculative Decoding](https://mlsanity.com/paper/2609.24150/) (2026)
- [BV Loss: Block Verification-Aware Loss for Block Diffusion Speculative Decoding](https://mlsanity.com/paper/2609.34832/) (2026)
- [Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Rounds](https://mlsanity.com/paper/2610.10411/) (2026)
- [TreeSpark: Learning to Draft the Trees You Verify for Speculative Decoding](https://mlsanity.com/paper/2610.20175/) (2026)

## Abstract (arXiv)

Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to propose candidate tokens that are then verified in parallel by the target model. The speedup is significantly determined by the acceptance rate, yet standard training minimizes Kullback-Leibler (KL) divergence as a proxy objective. While KL divergence and acceptance rate share the same global optimum, small draft models, having limited capacity, typically converge to suboptimal solutions where minimizing KL does not guarantee maximizing acceptance rate. To address this issue, we propose LK losses, special training objectives that directly target acceptance rate. Comprehensive experiments across four draft architectures and six target models, ranging from 8B to 685B parameters, demonstrate consistent improvements in acceptance metrics across all configurations compared to the standard KL-based training. We evaluate our approach on general, coding and math domains and report gains of up to 8-10% in average acceptance length. LK losses are easy to implement, introduce no computational overhead and can be directly integrated into any existing speculator training framework, making them a compelling alternative to the existing draft training objectives.