mlsanityData to 14 Oct 2026

Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction

arXiv 2504.15266 ↗First posted 21 Apr 2025cs.LG cs.AI cs.CLcited by 29

Not on the map. Why?

Abstract (arXiv)

We design a suite of minimal algorithmic tasks that are a loose abstraction of open-ended real-world tasks. This allows us to cleanly and controllably quantify the creative limits of the present-day language model. Much like real-world tasks that require a creative, far-sighted leap of thought, our tasks require an implicit, open-ended stochastic planning step that either (a) discovers new connections in an abstract knowledge graph (like in wordplay, drawing analogies, or research) or (b) constructs new patterns (like in designing math problems or new proteins). In these tasks, we empirically and conceptually argue how next-token learning is myopic; multi-token approaches, namely teacherless training and diffusion models, comparatively excel in producing diverse and original output. Secondly, to elicit randomness without hurting coherence, we find that injecting noise at the input layer (dubbed seed-conditioning) works surprisingly as well as (and in some conditions, better than) temperature sampling from the output layer. Thus, our work offers a principled, minimal test-bed for analyzing open-ended creative skills, and offers new arguments for going beyond next-token learning and temperature sampling. We make part of the code available under https://github.com/chenwu98/algorithmic-creativity

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 103 of its 161 references that are in the map. First posted on arXiv 21 Apr 2025.

Builds on

103 of 161 references.

References · 103

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2022Generative Adversarial Networkscited by 5,627
  5. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  7. 2020Extracting Training Data from Large Language Modelscited by 2,189
  8. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  9. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  10. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  11. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  12. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
38 more
  1. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
  2. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  3. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
  4. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  5. 2015Generating Sentences from a Continuous Spacecited by 612
  6. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  7. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
  8. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 498
  9. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 491
  10. 2017Non-Autoregressive Neural Machine Translationcited by 463
  11. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 450
  12. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  13. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 407
  14. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 382
  15. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 363
  16. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 360
  17. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 351
  18. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 312
  19. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspectivecited by 294
  20. 2022Transformers Learn Shortcuts to Automatacited by 268
  21. 2020Gradient Starvation: A Learning Proclivity in Neural Networkscited by 267
  22. 2017Noisy Networks for Explorationcited by 266
  23. 2023Think before you speak: Training Language Models With Pause Tokenscited by 264
  24. 2022Efficient Training of Language Models to Fill in the Middlecited by 255
  25. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 233
  26. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 178
  27. 2023Art or Artifice? Large Language Models and the False Promise of Creativitycited by 172
  28. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 169
  29. 2017Parameter Space Noise for Explorationcited by 161
  30. 2024The pitfalls of next-token predictioncited by 160
  31. 2023Protein Design with Guided Discrete Diffusioncited by 157
  32. 2023Physics of Language Models: Part 3.2, Knowledge Manipulationcited by 143
  33. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  34. 2024Homogenization Effects of Large Language Models on Human Creative Ideationcited by 127
  35. 2023Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventionscited by 125
  36. 2016Mode Regularized Generative Adversarial Networkscited by 123
  37. 2023Teaching Arithmetic to Small Transformerscited by 123
  38. 2023Calibrated Language Models Must Hallucinatecited by 115

50 most cited of 103 All 103 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 17.

29 papers.

Sort:

By area:

  1. 2025Jointly Reinforcing Diversity and Quality in Language Model Generationscited by 80
  2. 2025Next-Latent Prediction Transformers Learn Compact World Modelscited by 31
  3. 2025d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Cachingcited by 25
  4. 2026Language Modeling with Hyperspherical Flowscited by 22
  5. 2025Beyond Multi-Token Prediction: Pretraining LLMs with Future Summariescited by 15
  6. 2026On-Policy Self-Distillation with Sampled Demonstrations Reduces Output Diversitycited by 10
  7. 2026MLReplicate: Benchmarking Autonomous Research Systems for Machine Learning Reproducibilitycited by 8
  8. 2025Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scalingcited by 7
  9. 2025Measuring Combinatorial Creativity in Large Language Models and Humanscited by 7
  10. 2025Polychromic Objectives for Reinforcement Learningcited by 7
  11. 2026CREATE: Testing LLMs for Associative Creativitycited by 7
  12. 2025HypoSpace: A Diagnostic Benchmark for Set-Valued Hypothesis Generation under Underdetermination and Sublinear Coverage Boundscited by 6
17 more
  1. 2026Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalizationcited by 6
  2. 2026Free Lunch for Pass@k? Low Cost Diverse Sampling for Diffusion Language Modelscited by 5
  3. 2026Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Modelscited by 5
  4. 2026Causal Autoregressive Diffusion Language Modelcited by 3
  5. 2026Time-Annealed Perturbation Sampling: Diverse Generation for Diffusion Language Modelscited by 3
  6. 2025Internal Planning in Language Models: Characterizing Horizon and Branch Awarenesscited by 1
  7. 2025Leveraging AI multimodal geospatial foundation models for improved near-real-time flood mapping at a global scalecited by 1
  8. 2025Multiple Token Divergence: Measuring and Steering In-Context Computation Densitycited by 1
  9. 2026Hierarchical Latent Prediction for Language Modelscited by 1
  10. 2025How to model Human Actions distribution with Event Sequence Datacited by 0
  11. 2026SWE-Spot: Building Small Repo-Experts with Repository-Centric Learningcited by 0
  12. 2026Understanding the Emergence of Seemingly Useless Features in Next-Token Predictorscited by 0
  13. 2026Assessing the Creativity of Large Language Models: Testing, Limits, and New Frontierscited by 0
  14. 2026BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplerscited by 0
  15. 2026Sample What You Say: Aligning Language Models to Sample the Distributions They Statecited by 0
  16. 2026Blackboard Intelligence Can Surpass Autoregressive on Globally Constrained Problemscited by 0
  17. 2026Evaluating LLM Creativity through Long-Tail Performancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Diversity or Precision? A Deep Dive into Next Token Prediction
  2. 2026Noise Your Prompt: Noising Conditioning Tokens in Continuous Diffusion Language Models
  3. 2025On the Bias of Next-Token Predictors Toward Systematically Inefficient Reasoning: A Shortest-Path Case Study
  4. 2025How Reinforcement Learning After Next-Token Prediction Facilitates Learning
  5. 2026A Geometric Perspective on Next-Token Prediction in Large Language Models: Three Emerging Phases
  6. 2024Let's Think Dot by Dot: Hidden Computation in Transformer Language Models
  7. 2026Just on Time: Token-Level Early Stopping for Diffusion Language Models
  8. 2024Tokens, the oft-overlooked appetizer: Large language models, the distributional hypothesis, and meaning