mlsanityData to 14 Oct 2026

Parallel Token Prediction for Language Models

arXiv 2512.21323 ↗First posted 24 Dec 2025cs.CL cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Autoregressive decoding in language models is inherently slow, generating only one token per forward pass. We propose Parallel Token Prediction (PTP), a general-purpose framework for predicting multiple tokens in a single model call. PTP moves the source of randomness from post-hoc sampling to random input variables, making future tokens deterministic functions of those inputs and thus jointly predictable in a single forward pass. We prove that a single PTP call can represent arbitrary dependencies between tokens. PTP is trained by distilling an existing model or through inverse autoregressive training without a teacher. Experimentally, PTP achieves a 2.4x speedup on a diverse-task speculative decoding benchmark. We provide code and checkpoints at https://github.com/mandt-lab/ptp.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 38 of its 63 references that are in the map. First posted on arXiv 24 Dec 2025.

Builds on

38 of 63 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  6. 2017On Calibration of Modern Neural Networkscited by 4,960
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  8. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  9. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  10. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  11. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  12. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
26 more
  1. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  2. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  3. 2023StarCoder: may the source be with you!cited by 799
  4. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 640
  5. 2024TinyLlama: An Open-Source Small Language Modelcited by 618
  6. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 578
  7. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  8. 2017Non-Autoregressive Neural Machine Translationcited by 463
  9. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 370
  10. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 360
  11. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decodingcited by 298
  12. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decodingcited by 269
  13. 2020ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-trainingcited by 222
  14. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 215
  15. 2023SpecTr: Fast Speculative Decoding via Optimal Transportcited by 129
  16. 2024Hydra: Sequentially-Dependent Draft Heads for Medusa Decodingcited by 117
  17. 2019FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flowcited by 89
  18. 2023EasyTPP: Towards Open Benchmarking Temporal Point Processescited by 68
  19. 2024Discrete Copula Diffusioncited by 54
  20. 2024Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recyclingcited by 47
  21. 2025Theoretical Benefit and Limitation of Diffusion Language Modelcited by 47
  22. 2025Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potentialcited by 47
  23. 2024SAM Decoding: Speculative Decoding via Suffix Automatoncited by 39
  24. 2023Free-form Flows: Make Any Architecture a Normalizing Flowcited by 26
  25. 2025Speeding up Speculative Decoding via Sequential Approximate Verificationcited by 11
  26. 2024A Comprehensive Survey of Accelerated Generation Techniques in Large Language Modelscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026Multi-Token Prediction via Self-Distillationcited by 7
  2. 2026K-Forcing: Joint Next-K-Token Decoding via Push-Forward Language Modelingcited by 1
  3. 2026Conditional Total Correlation and the Serial Depth of Adaptive Parallel Samplingcited by 1
  4. 2026A Ticket from Marginals to Joints: Coupled-Noise Distillation for One-Step Block Generation in Diffusion Language Modelscited by 0
  5. 2026SEED: Self-Speculative Decoding via Implicit Encoder-Decodercited by 0
  6. 2026Gumbel Straight Flow: Distilling Autoregressive Models into One-step Flow Mapscited by 0
  7. 2026A Bethe-Ansatz Source of Early Token Correlations: A marginal-preserving construction for parallel proposalscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023PaSS: Parallel Speculative Sampling
  2. 2026AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding
  3. 2025Learning to Keep a Promise: Scaling Language Model Decoding Parallelism with Learned Asynchronous Decoding
  4. 2025Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decoding
  5. 2024ParallelSpec: Parallel Drafter for Efficient Speculative Decoding
  6. 2025Pre-Training Curriculum for Multi-Token Prediction in Language Models
  7. 2024Faster Language Models with Better Multi-Token Prediction Using Tensor Decomposition
  8. 2026MARS: Enabling Autoregressive Models Multi-Token Generation