# Accelerating Production LLMs with Combined Token/Embedding Speculators

arXiv: https://arxiv.org/abs/2404.19124 · HTML: https://mlsanity.com/paper/2404.19124/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving.
- Builds on 7 of its 13 references that are in the map.
- First posted on arXiv 29 Apr 2024.

## Placed in

- Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/)

## Builds on (most cited first, 7 of 7)

- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads](https://mlsanity.com/paper/2401.10774/) (2024)
- [EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty](https://mlsanity.com/paper/2401.15077/) (2024)
- [Break the Sequential Dependency of LLM Inference Using Lookahead Decoding](https://mlsanity.com/paper/2402.02057/) (2024)
- [Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding](https://mlsanity.com/paper/2402.05109/) (2024)
- [Recurrent Drafter for Fast Speculative Decoding in Large Language Models](https://mlsanity.com/paper/2403.09919/) (2024)
- [Multi-Candidate Speculative Decoding](https://mlsanity.com/paper/2401.06706/) (2024)

## Built on by (most cited first, 8 of 8)

- [Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment](https://mlsanity.com/paper/2501.19309/) (2025)
- [LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding](https://mlsanity.com/paper/2602.23881/) (2026)
- [Mixture of Attentions For Speculative Decoding](https://mlsanity.com/paper/2410.03804/) (2024)
- [Fast and Expressive Multi-Byte Prediction with Probabilistic Circuits](https://mlsanity.com/paper/2511.11346/) (2025)
- [MeshPad: Interactive Sketch-Conditioned Artist-Reminiscent Mesh Generation and Editing](https://mlsanity.com/paper/2503.01425/) (2025)
- [Reasoning Language Model Inference Serving Unveiled: An Empirical Study](https://mlsanity.com/paper/2510.18672/) (2025)
- [When Is a Draft Accepted? A Theory of Acceptance in Speculative Decoding](https://mlsanity.com/paper/2606.30265/) (2026)
- [LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter](https://mlsanity.com/paper/2609.37029/) (2026)

## Abstract (arXiv)

This technical report describes the design and training of novel speculative decoding draft models, for accelerating the inference speeds of large language models in a production environment. By conditioning draft predictions on both context vectors and sampled tokens, we can train our speculators to efficiently predict high-quality n-grams, which the base model then accepts or rejects. This allows us to effectively predict multiple tokens per inference forward pass, accelerating wall-clock inference speeds of highly optimized base model implementations by a factor of 2-3x. We explore these initial results and describe next steps for further improvements.