mlsanityData to 13 Oct 2026

Effective Reasoning Chains Reduce Intrinsic Dimensionality

arXiv 2602.09276 ↗First posted 9 Feb 2026cs.CL cs.AI cs.LGcited by 3

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Chain-of-thought (CoT) reasoning and its variants have substantially improved the performance of language models on complex reasoning tasks, yet the precise mechanisms by which different strategies facilitate generalization remain poorly understood. While current explanations often point to increased test-time computation or structural guidance, establishing a consistent, quantifiable link between these factors and generalization remains challenging. In this work, we identify intrinsic dimensionality as a quantitative measure for characterizing the effectiveness of reasoning chains. Intrinsic dimensionality quantifies the minimum number of model dimensions needed to reach a given accuracy threshold on a given task. By keeping the model architecture fixed and varying the task formulation through different reasoning strategies, we demonstrate that effective reasoning strategies consistently reduce the intrinsic dimensionality of the task. Validating this on GSM8K with Gemma-3 1B and 4B, we observe a strong inverse correlation between the intrinsic dimensionality of a reasoning strategy and its generalization performance on both in-distribution and out-of-distribution data. Our findings suggest that effective reasoning chains facilitate learning by better compressing the task using fewer parameters, offering a new quantitative metric for analyzing reasoning processes.

Key facts. Cited by 3 papers in the map (data through 13 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 37 of its 52 references that are in the map. First posted on arXiv 9 Feb 2026.

Builds on

37 of 52 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoT23 Sep
  2. 2025LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!11 Feb
  3. 2025When More is Less: Understanding Chain-of-Thought Length in LLMs11 Feb
  4. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning18 Sep
  5. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models6 May
  6. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters20 Dec
  7. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks22 Nov
  8. 2022Large Language Models are Zero-Shot Reasoners24 May
  9. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Models21 May
  10. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  11. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 26

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,916
  2. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,714
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 3,007
  4. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,954
  5. 2025Gemma 3 Technical Reportcited by 1,748
  6. 2025s1: Simple test-time scalingcited by 1,322
  7. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 964
  8. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 878
  9. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 873
  10. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 829
  11. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 665
  12. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 556
14 more
  1. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 548
  2. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 542
  3. 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 370
  4. 2022ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoningcited by 176
  5. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 126
  6. 2024Self-Discover: Large Language Models Self-Compose Reasoning Structurescited by 109
  7. 2025DeepSeek-R1 Thoughtology: Let's think about LLM Reasoningcited by 92
  8. 2025Reasoning with Sampling: Your Base Model is Smarter Than You Thinkcited by 86
  9. 2023ReCEval: Evaluating Reasoning Chains via Correctness and Informativenesscited by 76
  10. 2018The Description Length of Deep Learning Modelscited by 71
  11. 2024What is Wrong with Perplexity for Long-context Language Modeling?cited by 68
  12. 2025On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Modelscited by 67
  13. 2023Non-Vacuous Generalization Bounds for Large Language Modelscited by 53
  14. 2025Bridging Kolmogorov Complexity and Deep Learning: Asymptotically Optimal Description Length Objectives for Transformerscited by 6

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026On the Role of Reasoning Patterns in the Generalization Discrepancy of Long Chain-of-Thought Supervised Fine-Tuningcited by 4
  2. 2026The Scaling Properties of Implicit Deductive Reasoning in Transformerscited by 1
  3. 2026Geometric Signatures of Reasoning: A Spectral Perspective on Task Hardnesscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026An Asymptotic Theory of Chain-of-Thought in In-Context Learning
  2. 2025The relationship between reasoning and performance in large language models--o3 (mini) thinks harder, not longer
  3. 2025Beyond Chains of Thought: Benchmarking Latent-Space Reasoning Abilities in Large Language Models
  4. 2025Understanding Chain-of-Thought in Large Language Models via Topological Data Analysis
  5. 2025Deep Hidden Cognition Facilitates Reliable Chain-of-Thought Reasoning
  6. 2025Why Does Reasoning Length Converge? Unveiling the Underfitting-Overfitting Trade-off in Chain-of-Thought
  7. 2025What Makes a Good Reasoning Chain? Uncovering Structural Patterns in Long Chain-of-Thought Reasoning
  8. 2024Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs