mlsanityData to 7 Oct 2026

TokenTiming: A Dynamic Alignment Method for Universal Speculative Decoding Model Pairs

arXiv 2510.15545 ↗First posted 17 Oct 2025cs.CL cs.AIcited by 0

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Accelerating the inference of large language models (LLMs) has been a critical challenge in generative AI. Speculative decoding (SD) substantially improves LLM inference efficiency. However, its utility is limited by a fundamental constraint: the draft and target models must share the same vocabulary, thus limiting the herd of available draft models and often necessitating the training of a new model from scratch. Inspired by Dynamic Time Warping (DTW), a classic algorithm for aligning time series, we propose the algorithm TokenTiming for universal speculative decoding. It operates by re-encoding the draft token sequence to get a new target token sequence, and then uses DTW to build a mapping to transfer the probability distributions for speculative sampling. Benefiting from this, our method accommodates mismatched vocabularies and works with any off-the-shelf models without retraining and modification. We conduct comprehensive experiments on various tasks, demonstrating 1.57x speedup. This work enables a universal approach for draft model selection, making SD a more versatile and practical tool for LLM acceleration.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 21 of its 33 references that are in the map. First posted on arXiv 17 Oct 2025.

Builds on

21 of 33 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025Out-of-Vocabulary Sampling Boosts Speculative Decoding2 Jun
  2. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test3 Mar
  3. 2025Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies31 Jan
  4. 2024MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding20 Aug
  5. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees24 Jun
  6. 2024Distributed Speculative Inference (DSI): Speculation Parallelism for Provably Faster Lossless Language Model Inference23 May
  7. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decoding3 Feb
  8. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  9. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding15 Jan
  10. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification16 May
  11. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  12. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 9

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  5. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  7. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  8. 2019Neural Machine Translation with Byte-Level Subwordscited by 115
  9. 2019Long and Diverse Text Generation with Planning-based Hierarchical Variational Modelcited by 62

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025GRIFFIN: Effective Token Alignment for Faster Speculative Decoding
  2. 2024Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding
  3. 2025AdaSD: Adaptive Speculative Decoding for Efficient Language Model Inference
  4. 2025Alignment-Augmented Speculative Decoding with Alignment Sampling and Conditional Verification
  5. 2025Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment
  6. 2024Recursive Speculative Decoding: Accelerating LLM Inference via Sampling Without Replacement
  7. 2025READER: Retrieval-Assisted Drafter for Efficient LLM Inference
  8. 2025Tutorial Proposal: Speculative Decoding for Efficient LLM Inference