mlsanityData to 7 Oct 2026

The Illusion of Stochasticity in LLMs

arXiv 2604.06543 ↗First posted 8 Apr 2026cs.CL cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

In this work, we demonstrate that reliable stochastic sampling is a fundamental yet unfulfilled requirement for Large Language Models (LLMs) operating as agents. Agentic systems are frequently required to sample from distributions, often inferred from observed data, a process which needs to be emulated by the LLM. This leads to a distinct failure point: while standard RL agents rely on external sampling mechanisms, LLMs fail to map their internal probability estimates to their stochastic outputs. Through rigorous empirical analysis across multiple model families, model sizes, prompting styles, and distributions, we demonstrate the extent of this failure. Crucially, we show that while powerful frontier models can convert provided random seeds to target distributions, their ability to sample directly from specific distributions is fundamentally flawed.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 32 of its 43 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

32 of 43 references.

References · 32

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  7. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  8. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 564
  9. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  10. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 141
  11. 2024Tokenization counts: the impact of tokenization on arithmetic in frontier LLMscited by 112
  12. 2025Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversitycited by 100
20 more
  1. 2023Amortizing intractable inference in large language modelscited by 94
  2. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 93
  3. 2024Transformers need glasses! Information over-squashing in language taskscited by 87
  4. 2025Inverse Scaling in Test-Time Computecited by 49
  5. 2024Forcing Diffuse Distributions out of Language Modelscited by 48
  6. 2024Number Cookbook: Number Understanding of Language Models and How to Improve Itcited by 41
  7. 2025LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilitiescited by 36
  8. 2024EVOLvE: Evaluating and Optimizing LLMs For In-Context Explorationcited by 33
  9. 2024LMAct: A Benchmark for In-Context Imitation Learning with Long Multimodal Demonstrationscited by 32
  10. 2024The CLRS-Text Algorithmic Reasoning Language Benchmarkcited by 28
  11. 2024Heads, Tails, and AI Fails: LLMs, Randomness, and Human Judgmentscited by 23
  12. 2025Enough Coin Flips Can Make LLMs Act Bayesiancited by 19
  13. 2024Do LLMs Play Dice? Exploring Probability Distribution Sampling in Large Language Models for Behavioral Simulationcited by 14
  14. 2024Fourier Head: Helping Large Language Models Learn Complex Probability Distributionscited by 10
  15. 2024A Comparison of Large Language Model and Human Performance on Random Number Generation Taskscited by 7
  16. 2025Playing games with Large language models: Randomness and strategycited by 7
  17. 2025Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Samplingcited by 7
  18. 2025Efficient numeracy in language models through single-token number embeddingscited by 6
  19. 2026Large Language Models Are Bad Dice Players: LLMs Struggle to Generate Random Numbers from Statistical Distributionscited by 5
  20. 2024Learning Randomized Algorithms with Transformerscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026Probabilistic Calibration Is a Trainable Capability in Language Modelscited by 2
  2. 2026Partition, Prompt, Aggregate: Statistical Self-Consistency in Language Modelscited by 2
  3. 2026Prior Knowledge or Search? A Study of LLM Agents in Hardware-Aware Code Optimizationcited by 1
  4. 2026Structured Inference with Large Language Gibbscited by 0
  5. 2026Mitigating Scoring Bias in LLM-as-a-Judge via Random Number Generationcited by 0
  6. 2026Sample What You Say: Aligning Language Models to Sample the Distributions They Statecited by 0
  7. 2026Shared and structured inputs undermine collective random choice by reasoning AI agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Recipes for Steering and Scaling LLMs via Sampling
  2. 2025Large Language Models as Discounted Bayesian Filters
  3. 2024Practical Considerations for Agentic LLM Systems
  4. 2025Reasoning Under Uncertainty: Exploring Probabilistic Reasoning Capabilities of LLMs
  5. 2025LMR-BENCH: Evaluating LLM Agent's Ability on Reproducing Language Modeling Research
  6. 2025Knowing What You Know Is Not Enough: Large Language Model Confidences Don't Align With Their Actions
  7. 2026Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Study
  8. 2024Learning From Failure: Integrating Negative Examples when Fine-tuning Large Language Models as Agents