# LLMs are Not Just Next Token Predictors

arXiv: https://arxiv.org/abs/2408.04666 · HTML: https://mlsanity.com/paper/2408.04666/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 13 references that are in the map.
- First posted on arXiv 6 Aug 2024.

## Builds on (most cited first, 1 of 1)

- [Talking About Large Language Models](https://mlsanity.com/paper/2212.03551/) (2022)

## Built on by (most cited first, 3 of 3)

- [Still "Talking About Large Language Models": Some Clarifications](https://mlsanity.com/paper/2412.10291/) (2024)
- [Deflating Deflationism: A Critical Perspective on Debunking Arguments Against LLM Mentality](https://mlsanity.com/paper/2506.13403/) (2025)
- [Constructive Apraxia: An Unexpected Limit of Instructible Vision-Language Models and Analog for Human Cognitive Disorders](https://mlsanity.com/paper/2410.03551/) (2024)

## Abstract (arXiv)

LLMs are statistical models of language learning through stochastic gradient descent with a next token prediction objective. Prompting a popular view among AI modelers: LLMs are just next token predictors. While LLMs are engineered using next token prediction, and trained based on their success at this task, our view is that a reduction to just next token predictor sells LLMs short. Moreover, there are important explanations of LLM behavior and capabilities that are lost when we engage in this kind of reduction. In order to draw this out, we will make an analogy with a once prominent research program in biology explaining evolution and development from the gene's eye view.