Reasoning emerges from constrained inference manifolds in large language models
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Reasoning in large language models is predominantly evaluated through labeled benchmarks, conflating task performance with the quality of internal inference. Here we study reasoning as an intrinsic dynamical process by examining the evolution of internal representations during inference. We find that inference-time dynamics consistently self-organize into low-dimensional manifolds embedded within high-dimensional representation spaces. we find that such geometric compression, although pervasive, is not sufficient for stable or reliable reasoning. Instead, effective reasoning dynamics emerge within a constrained structural regime characterized by three conditions: adequate representational expressivity, spontaneous manifold compression, and preservation of non-degenerate information volume within the compressed subspace. Models outside this regime exhibit characteristic pathological inference dynamics. Based on these insights, we introduce a unified, label-free diagnostic computed solely from internal dynamics. These findings suggest that reasoning in LLMs is fundamentally governed by geometric and informational constraints, offering a complementary framework to benchmark-centric assessment.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 23 of its 27 references that are in the map. First posted on arXiv 2 May 2026.
Builds on
23 of 27 references.
Closest ancestors · 2
Same direction, newest first
- 2025Qwen3 Technical Report14 May
- 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey2 Apr
Other references · 21
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2024GPT-4o System Cardcited by 4,467
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
- 2022Toy Models of Superpositioncited by 928
- 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
- 2019Intrinsic dimension of data representations in deep neural networkscited by 276
- 2018Estimating the intrinsic dimension of datasets by a minimal neighborhood informationcited by 246
9 more
- 2024On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Modelscited by 42
- 2024Densing Law of LLMscited by 39
- 2025Mechanistic understanding and validation of large AI models with SemanticLenscited by 29
- 2025Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarkscited by 26
- 2023Predicting and Enhancing the Fairness of DNNs with the Curvature of Perceptual Manifoldscited by 9
- 2025Geometry of Decision Making in Language Modelscited by 8
- 2023Inversion dynamics of class manifolds in deep learning reveals tradeoffs underlying generalisationcited by 6
- 2024Unveiling and Mitigating Generalized Biases of DNNs through the Intrinsic Dimensions of Perceptual Manifoldscited by 6
- 2024Sampling-enabled scalable manifold learning unveils the discriminative cluster structure of high-dimensional datacited by 3
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026FutureNav: Unified World-Action Modeling for Vision-and-Language Navigationcited by 5
- 2026GeoFaith: A Spatio-Temporal Dual View of Faithful Chain-of-Thoughtcited by 0
- 2026Hybrid NARX-LLM for Greenland Iceberg Discharge: Prompt-Driven Residual Correctioncited by 0
- 2026From Brewing to Resolution: Tracing the Internal Lifecycle of Code Reasoning in LLMscited by 0
- 2026Geometric Similarity in VLM Low-Level Vision Representationscited by 0
- 2026Geometric Similarity in VLM Low-Level Vision Representations1 Oct 2026
- 2026FutureNav: Unified World-Action Modeling for Vision-and-Language Navigation29 Jun 2026
- 2026From Brewing to Resolution: Tracing the Internal Lifecycle of Code Reasoning in LLMs16 Jun 2026
- 2026Hybrid NARX-LLM for Greenland Iceberg Discharge: Prompt-Driven Residual Correction13 Jun 2026
- 2026GeoFaith: A Spatio-Temporal Dual View of Faithful Chain-of-Thought26 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026Emergent Manifold Separability during Reasoning in Large Language Models
- 2026The Geometric Price of Discrete Logic: Context-driven Manifold Dynamics of Number Representations
- 2026Reasoning as State Transition: A Representational Analysis of Reasoning Evolution in Large Language Models
- 2026Unveiling the Reasoning Process of Large Language Models
- 2026Dissecting Failure Dynamics in Large Language Model Reasoning
- 2025The Geometry of Reasoning: Flowing Logics in Representation Space
- 2026Spatiotemporal Hidden-State Dynamics as a Signature of Internal Reasoning in Large Language Models
- 2025REMA: A Unified Reasoning Manifold Framework for Interpreting Large Language Model