mlsanityData to 7 Oct 2026

The pitfalls of next-token prediction

arXiv 2403.06963 ↗First posted 11 Mar 2024cs.CL cs.AI cs.LGcited by 156

Not on the map. Why?

Cited by 38 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Can a mere next-token predictor faithfully model human intelligence? We crystallize this emerging concern and correct popular misconceptions surrounding it, and advocate a simple multi-token objective. As a starting point, we argue that the two often-conflated phases of next-token prediction -- autoregressive inference and teacher-forced training -- must be treated distinctly. The popular criticism that errors can compound during autoregressive inference, crucially assumes that teacher-forcing has learned an accurate next-token predictor. This assumption sidesteps a more deep-rooted problem we expose: in certain classes of tasks, teacher-forcing can simply fail to learn an accurate next-token predictor in the first place. We describe a general mechanism of how teacher-forcing can fail, and design a minimal planning task where both the Transformer and the Mamba architecture empirically fail in that manner -- remarkably, despite the task being straightforward to learn. Finally, we provide preliminary evidence that this failure can be resolved using _teacherless_ training, a simple modification using dummy tokens that predicts multiple tokens in advance. We hope this finding can ground future debates and inspire explorations beyond the next-token prediction paradigm. We make our code available under https://github.com/gregorbachmann/Next-Token-Failures

Key facts. Cited by 156 papers in the map (data through 7 Oct 2026). Builds on 79 of its 119 references that are in the map. First posted on arXiv 11 Mar 2024.

Builds on

79 of 119 references.

References · 79

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  7. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  9. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  10. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  11. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  12. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
38 more
  1. 2020Learning to summarize from human feedbackcited by 2,562
  2. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  3. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  4. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  5. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  6. 2022In-context Learning and Induction Headscited by 930
  7. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  8. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  9. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  10. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  11. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  12. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  13. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  14. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  15. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  16. 2017Non-Autoregressive Neural Machine Translationcited by 457
  17. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
  18. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 372
  19. 2017A Deep Reinforced Model for Abstractive Summarizationcited by 364
  20. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 349
  21. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  22. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 318
  23. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 308
  24. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspectivecited by 293
  25. 2020Gradient Starvation: A Learning Proclivity in Neural Networkscited by 262
  26. 2022Transformers Learn Shortcuts to Automatacited by 260
  27. 2023Think before you speak: Training Language Models With Pause Tokenscited by 254
  28. 2020ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-trainingcited by 221
  29. 2016Professor Forcing: A New Algorithm for Training Recurrent Networkscited by 214
  30. 2022Exploring Length Generalization in Large Language Modelscited by 204
  31. 2022The Parallelism Tradeoff: Limitations of Log-Precision Transformerscited by 191
  32. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 175
  33. 2016An Actor-Critic Algorithm for Sequence Predictioncited by 158
  34. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 158
  35. 2022Distilling Reasoning Capabilities into Smaller Language Modelscited by 155
  36. 2023Physics of Language Models: Part 3.2, Knowledge Manipulationcited by 142
  37. 2023Teaching Arithmetic to Small Transformerscited by 121
  38. 2014Reinforcement and Imitation Learning via Interactive No-Regret Learningcited by 116

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 36; 2025: 72; 2026: 48.

156 papers.

Sort:

By area:

  1. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  2. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
  3. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  4. 2024Stream of Search (SoS): Learning to Search in Languagecited by 141
  5. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  6. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 118
  7. 2024RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Foldcited by 99
  8. 2025Dream-Coder 7B: An Open Diffusion Language Model for Codecited by 81
  9. 2024Transformers represent belief state geometry in their residual streamcited by 73
  10. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  11. 2024State Space Model for New-Generation Network Alternative to Transformers: A Surveycited by 59
  12. 2025Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignmentcited by 59
38 more
  1. 2025FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocitiescited by 46
  2. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 38
  3. 2025LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMscited by 38
  4. 2025Rethinking External Slow-Thinking: From Snowball Errors to Probability of Correct Reasoningcited by 36
  5. 2024System-1.x: Learning to Balance Fast and Slow Planning with Language Modelscited by 35
  6. 2024Causal Language Modeling Can Elicit Search and Reasoning Capabilities on Logic Puzzlescited by 35
  7. 2025ReVISE: Learning to Refine at Test-Time via Intrinsic Self-Verificationcited by 35
  8. 2025LLM Pretraining with Continuous Conceptscited by 33
  9. 2024How Susceptible are LLMs to Influence in Prompts?cited by 32
  10. 2025Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challengescited by 30
  11. 2025TESS 2: A Large-Scale Generalist Diffusion Language Modelcited by 30
  12. 2025Next-Latent Prediction Transformers Learn Compact World Modelscited by 29
  13. 2025Roll the dice & look before you leap: Going beyond the creative limits of next-token predictioncited by 28
  14. 2024Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?cited by 26
  15. 2025Cognitive Foundations for Reasoning and Their Manifestation in LLMscited by 26
  16. 2024What Are the Odds? Language Models Are Capable of Probabilistic Reasoningcited by 25
  17. 2024Non-myopic Generation of Language Models for Reasoning and Planningcited by 25
  18. 2025LaDiR: Latent Diffusion Enhances LLMs for Text Reasoningcited by 25
  19. 2024Navigating Shortcuts, Spurious Correlations, and Confounders: From Origins via Detection to Mitigationcited by 22
  20. 2024Towards Trustworthy AI: A Review of Ethical and Robust Large Language Modelscited by 21
  21. 2024Imitating Language via Scalable Inverse Reinforcement Learningcited by 21
  22. 2024Transformers Struggle to Learn to Searchcited by 19
  23. 2025DLM-One: Diffusion Language Models for One-Step Sequence Generationcited by 19
  24. 2025Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbonecited by 18
  25. 2025Emergent Response Planning in LLMscited by 17
  26. 2024Reinforcing Language Agents via Policy Optimization with Action Decompositioncited by 16
  27. 2025Multi-Token Prediction Needs Registerscited by 16
  28. 2024Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoningcited by 15
  29. 2025Exposing Numeracy Gaps: A Benchmark to Evaluate Fundamental Numerical Abilities in Large Language Modelscited by 14
  30. 2025Qsharp: Provably Optimal Distributional RL for LLM Post-Trainingcited by 14
  31. 2025Beyond Multi-Token Prediction: Pretraining LLMs with Future Summariescited by 14
  32. 2024Frontiers in Intelligent Colonoscopycited by 13
  33. 2025Have Large Language Models Learned to Reason? A Characterization via 3-SAT Phase Transitioncited by 13
  34. 2025Enhancing Latent Computation in Transformers with Latent Tokenscited by 13
  35. 2024The Belief State Transformercited by 12
  36. 2024Regress, Don't Guess -- A Regression-like Loss on Number Tokens for Language Modelscited by 12
  37. 2025Computational-Statistical Tradeoffs at the Next-Token Prediction Barrier: Autoregressive and Imitation Learning under Misspecificationcited by 12
  38. 2026Continuous Latent Diffusion Language Modelcited by 12

50 most cited of 156 All 156 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Blackboard Intelligence Can Surpass Autoregressive on Globally Constrained Problems
  2. 2025How Reinforcement Learning After Next-Token Prediction Facilitates Learning
  3. 2025On the Bias of Next-Token Predictors Toward Systematically Inefficient Reasoning: A Shortest-Path Case Study
  4. 2025Internal states before wait modulate reasoning patterns
  5. 2025From Next Token Prediction to (STRIPS) World Models
  6. 2025Scaling Competence, Shrinking Reasoning: Cognitive Signatures in Language Model Learning
  7. 2026Match the Distribution, Not the Compute: Post-Training Multi-Token Prediction Heads
  8. 2025Diversity or Precision? A Deep Dive into Next Token Prediction