mlsanityData to 14 Oct 2026

Intuition emerges in Maximum Caliber models at criticality

arXiv 2508.06477 ↗First posted 8 Aug 2025physics.soc-ph cond-mat.dis-nn cond-mat.stat-mechcited by 0

Not on the map. Why?

Abstract (arXiv)

Whether large predictive models merely parrot their training data or produce genuine insight lacks a physical explanation. This work reports a primitive form of intuition that emerges as a metastable phase of learning that critically balances next-token prediction against future path-entropy. The intuition mechanism is discovered via mind-tuning, the minimal principle that imposes Maximum Caliber in predictive models with a control temperature-like parameter λ. Training on random walks in deterministic mazes reveals a rich phase diagram: imitation (low λ), rule-breaking hallucination (high λ), and a fragile in-between window exhibiting strong protocol-dependence (hysteresis) and multistability, where models spontaneously discover novel goal-directed strategies. These results are captured by an effective low-dimensional theory and frame intuition as an emergent property at the critical balance between memorizing what is and wondering what could be.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 53 references that are in the map. First posted on arXiv 8 Aug 2025.

Builds on

26 of 53 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  7. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  8. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  9. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 867
  10. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
  11. 2025Small Language Models are the Future of Agentic AIcited by 270
  12. 2020The large learning rate phase of deep learning: the catapult mechanismcited by 216
14 more
  1. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 176
  2. 2022The Debate Over Understanding in AI's Large Language Modelscited by 142
  3. 2015Statistical physics of inference: Thresholds and algorithmscited by 128
  4. 2024Superhuman performance of a large language model on the reasoning tasks of a physiciancited by 28
  5. 2022Designing Ecosystems of Intelligence from First Principlescited by 23
  6. 2024Intelligence at the Edge of Chaoscited by 13
  7. 2025Parameter Symmetry Potentially Unifies Deep Learning Theorycited by 12
  8. 2023Collective behavior from surprise minimizationcited by 10
  9. 2025Neural Thermodynamics: Entropic Forces in Deep and Universal Representation Learningcited by 9
  10. 2022Complex behavior from intrinsic motivation to occupy action-state path spacecited by 7
  11. 2023An effective theory of collective deep learningcited by 7
  12. 2025Leveraging chaotic transients in the training of artificial neural networkscited by 3
  13. 2025Intrinsic motivation as constrained entropy maximizationcited by 2
  14. 2025The Missing Reward: Active Inference in the Era of Experiencecited by 2

Built on by

Not cited yet.