mlsanityData to 7 Oct 2026

Mean-Field Dynamics of Chain-of-Thought Reasoning in Large Language Models

arXiv 2608.05152 ↗First posted 20 May 2026cs.CL cs.AIcited by 0

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large language models (LLMs) with chain-of-thought reasoning have been widely applied in recent years, and theoretical explanations of their behavior may help deepen our understanding and guide model optimization. In this study, we introduce a framework that seeks statistical regularities and theoretical interpretations in LLM reasoning without simplifying the model architecture or making analogies to existing physical systems. We formulate LLM reasoning as a guided discovery process on a clue graph, and derive a one-dimensional ordinary differential equation for the fraction of discovered clues using the mean-field approximation. Experimentally, clue tokens are identified using the normalized surprisal of a student LLM on the outputs of a teacher LLM, and statistical regularities are obtained by averaging over many reasoning chains of thought. Our experiments show that the resulting statistical regularities are reproducible within the same dataset and can be fitted by the solving the proposed theoretical equation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 40 references that are in the map. First posted on arXiv 20 May 2026.

Builds on

34 of 40 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  5. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  6. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  7. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  8. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  9. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  10. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  11. 2022In-context Learning and Induction Headscited by 930
  12. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
20 more
  1. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
  2. 2016Sequence-Level Knowledge Distillationcited by 761
  3. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  4. 2022Transformers learn in-context by gradient descentcited by 714
  5. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  6. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
  7. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 446
  8. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  9. 2022FOLIO: Natural Language Reasoning with First-Order Logiccited by 233
  10. 2023MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoningcited by 195
  11. 2021The Principles of Deep Learning Theorycited by 169
  12. 2019CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Textcited by 155
  13. 2024Finding Transformer Circuits with Edge Pruningcited by 58
  14. 2024A phase transition between positional and semantic learning in a solvable model of dot-product attentioncited by 30
  15. 2022Grokking phase transitions in learning local rules with gradient descentcited by 28
  16. 2025Language Models Grow Less Humanlike beyond Phase Transitioncited by 10
  17. 2025Advanced Financial Reasoning at Scale: A Comprehensive Evaluation of Large Language Models on CFA Level IIIcited by 6
  18. 2024Spin glass model of in-context learningcited by 2
  19. 2024Berezinskii--Kosterlitz--Thouless transition in a context-sensitive random language modelcited by 1
  20. 2026LogitScope: A Framework for Analyzing LLM Uncertainty Through Information Metricscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Metastable Dynamics of Chain-of-Thought Reasoning: Provable Benefits of Search, RL and Distillation
  2. 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance
  3. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
  4. 2026How Well Do LLMs Perform on the Simplest Long-Chain Reasoning Tasks: An Empirical Study on the Equivalence Class Problem
  5. 2023Why Can Large Language Models Generate Correct Chain-of-Thoughts?
  6. 2025Early Stopping Chain-of-thoughts in Large Language Models
  7. 2023Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models
  8. 2024Chain-of-Thought in Large Language Models: Decoding, Projection, and Activation