mlsanityData to 7 Oct 2026

Emergent Manifold Separability during Reasoning in Large Language Models

arXiv 2602.20338 ↗First posted 23 Feb 2026cs.LGcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Chain-of-Thought (CoT) prompting significantly improves reasoning in Large Language Models, yet the temporal dynamics of the underlying representation geometry remain poorly understood. We investigate these dynamics by applying Manifold Capacity Theory (MCT) to two compositional reasoning tasks: a controlled Boolean logic tree that supports deep mechanistic analysis, and a natural-language eligibility task in which the model has to extract attributes from prose, compare them to thresholds, and compose the local decisions through a fixed evaluation tree. MCT lets us quantify the linear separability of latent representations without the confounding factors of probe training. On both tasks, and across several open-weight models, reasoning manifests as a transient geometric pulse: concept manifolds are untangled into linearly separable subspaces immediately prior to computation and rapidly compressed thereafter. This behavior diverges from standard linear probe accuracy, which remains high long after computation, suggesting a fundamental distinction between information that is merely retrievable and information that is geometrically prepared for processing. We interpret this phenomenon as Dynamic Manifold Management, a mechanism where the model dynamically modulates representational capacity to optimize the bandwidth of the residual stream throughout the reasoning chain.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 24 of its 32 references that are in the map. First posted on arXiv 23 Feb 2026.

Builds on

24 of 32 references.

References · 24

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  3. 2022In-context Learning and Induction Headscited by 930
  4. 2022Toy Models of Superpositioncited by 928
  5. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  6. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  7. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  8. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  9. 2019Designing and Interpreting Probes with Control Taskscited by 608
  10. 2018Insights on representational similarity in neural networks with canonical correlationcited by 318
  11. 2021The Intrinsic Dimension of Images and Its Impact on Learningcited by 307
  12. 2023Language Models Represent Space and Timecited by 306
12 more
  1. 2019Intrinsic dimension of data representations in deep neural networkscited by 276
  2. 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verificationcited by 154
  3. 2021On the geometry of generalization and memorization in deep neural networkscited by 71
  4. 2017Classification and Geometry of General Perceptual Manifoldscited by 62
  5. 2025Unveiling the Key Factors for Distilling Chain-of-Thought Reasoningcited by 40
  6. 2020Emergence of Separable Manifolds in Deep Language Representationscited by 39
  7. 2019No Training Required: Exploring Random Encoders for Sentence Classificationcited by 30
  8. 2025Knowing Before Saying: LLM Representations Encode Information About Chain-of-Thought Success Before Completioncited by 22
  9. 2018Analyzing biological and artificial neural networks: challenges with opportunities for synergy?cited by 20
  10. 2018Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysiscited by 17
  11. 2025The Geometry of Prompting: Unveiling Distinct Mechanisms of Task Adaptation in Language Modelscited by 14
  12. 2020Untangling in Invariant Speech Recognitioncited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Reasoning emerges from constrained inference manifolds in large language models
  2. 2026The Geometric Price of Discrete Logic: Context-driven Manifold Dynamics of Number Representations
  3. 2025Hypothesis-Driven Feature Manifold Analysis in LLMs via Supervised Multi-Dimensional Scaling
  4. 2025REMA: A Unified Reasoning Manifold Framework for Interpreting Large Language Model
  5. 2026Unveiling the Reasoning Process of Large Language Models
  6. 2026The Geometry of Thought: How Scale Restructures Reasoning In Large Language Models
  7. 2025The Geometry of Reasoning: Flowing Logics in Representation Space
  8. 2026Reasoning on the Manifold: Bidirectional Consistency for Self-Verification in Diffusion Language Models