# Multi-token prediction

Area: [LLM pre-training data and scaling laws](https://mlsanity.com/pretrain/index.md). HTML: https://mlsanity.com/pretrain/multi-token-prediction/

## Key facts

- Multi-token prediction: 19 papers on arXiv since 2024 (data through 19 Oct 2026).
- 17 of its 19 mapped papers (89%) cite “Better & Faster Large Language Models via Multi-token Prediction” (2024, arXiv 2404.19737).
- 4 of its 19 mapped papers (21%) cite “Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potential” (2025, arXiv 2507.11851).
- 2 new papers 1 Oct–19 Oct.

## Most cited here

- [Better & Faster Large Language Models via Multi-token Prediction](https://mlsanity.com/paper/2404.19737/) (2024), cited by 17 papers here
- [Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potential](https://mlsanity.com/paper/2507.11851/) (2025), cited by 4 papers here
- [Efficient Joint Prediction of Multiple Future Tokens](https://mlsanity.com/paper/2503.21801/) (2025), cited by 3 papers here
- [Multi-Token Prediction Needs Registers](https://mlsanity.com/paper/2505.10518/) (2025), cited by 3 papers here
- [L-MTP: Leap Multi-Token Prediction Beyond Adjacent Context for Large Language Models](https://mlsanity.com/paper/2505.17505/) (2025), cited by 3 papers here
- [Understanding and Enhancing the Planning Capability of Language Models via Multi-Token Prediction](https://mlsanity.com/paper/2509.23186/) (2025), cited by 2 papers here
- [Beyond Multi-Token Prediction: Pretraining LLMs with Future Summaries](https://mlsanity.com/paper/2510.14751/) (2025), cited by 2 papers here
- [Faster Language Models with Better Multi-Token Prediction Using Tensor Decomposition](https://mlsanity.com/paper/2410.17765/) (2024), cited by 1 papers here
- [LLM Pretraining with Continuous Concepts](https://mlsanity.com/paper/2502.08524/) (2025), cited by 1 papers here
- [Predicting the Order of Upcoming Tokens Improves Language Modeling](https://mlsanity.com/paper/2508.19228/) (2025), cited by 1 papers here

## New (1 Oct–19 Oct)

- [ECHO: Efficient Causal Self-Recursion Beyond Auxiliary Multi-Token Prediction](https://mlsanity.com/paper/2610.24766/) (2026-10-17)
- [LightMTP: Lightweight Latent Multi-Token Prediction](https://mlsanity.com/paper/2610.06031/) (2026-10-05)