mlsanityData to 7 Oct 2026

Reasoning emerges from constrained inference manifolds in large language models

arXiv 2605.08142 ↗First posted 2 May 2026cs.LG cs.CL cs.CVcited by 5

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Reasoning in large language models is predominantly evaluated through labeled benchmarks, conflating task performance with the quality of internal inference. Here we study reasoning as an intrinsic dynamical process by examining the evolution of internal representations during inference. We find that inference-time dynamics consistently self-organize into low-dimensional manifolds embedded within high-dimensional representation spaces. we find that such geometric compression, although pervasive, is not sufficient for stable or reliable reasoning. Instead, effective reasoning dynamics emerge within a constrained structural regime characterized by three conditions: adequate representational expressivity, spontaneous manifold compression, and preservation of non-degenerate information volume within the compressed subspace. Models outside this regime exhibit characteristic pathological inference dynamics. Based on these insights, we introduce a unified, label-free diagnostic computed solely from internal dynamics. These findings suggest that reasoning in LLMs is fundamentally governed by geometric and informational constraints, offering a complementary framework to benchmark-centric assessment.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 23 of its 27 references that are in the map. First posted on arXiv 2 May 2026.

Builds on

23 of 27 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Qwen3 Technical Report14 May
  2. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey2 Apr

Other references · 21

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2024GPT-4o System Cardcited by 4,467
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  6. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  7. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  8. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  9. 2022Toy Models of Superpositioncited by 928
  10. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  11. 2019Intrinsic dimension of data representations in deep neural networkscited by 276
  12. 2018Estimating the intrinsic dimension of datasets by a minimal neighborhood informationcited by 246
9 more
  1. 2024On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Modelscited by 42
  2. 2024Densing Law of LLMscited by 39
  3. 2025Mechanistic understanding and validation of large AI models with SemanticLenscited by 29
  4. 2025Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarkscited by 26
  5. 2023Predicting and Enhancing the Fairness of DNNs with the Curvature of Perceptual Manifoldscited by 9
  6. 2025Geometry of Decision Making in Language Modelscited by 8
  7. 2023Inversion dynamics of class manifolds in deep learning reveals tradeoffs underlying generalisationcited by 6
  8. 2024Unveiling and Mitigating Generalized Biases of DNNs through the Intrinsic Dimensions of Perceptual Manifoldscited by 6
  9. 2024Sampling-enabled scalable manifold learning unveils the discriminative cluster structure of high-dimensional datacited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Emergent Manifold Separability during Reasoning in Large Language Models
  2. 2026The Geometric Price of Discrete Logic: Context-driven Manifold Dynamics of Number Representations
  3. 2026Reasoning as State Transition: A Representational Analysis of Reasoning Evolution in Large Language Models
  4. 2026Unveiling the Reasoning Process of Large Language Models
  5. 2026Dissecting Failure Dynamics in Large Language Model Reasoning
  6. 2025The Geometry of Reasoning: Flowing Logics in Representation Space
  7. 2026Spatiotemporal Hidden-State Dynamics as a Signature of Internal Reasoning in Large Language Models
  8. 2025REMA: A Unified Reasoning Manifold Framework for Interpreting Large Language Model