Hierarchical Chain-of-Thought: Enhancing LLM Reasoning Performance and Efficiency
On the map: LLM reasoning → Prompting and chain-of-thought reasoning
Abstract (arXiv)
Chain-of-Thought (CoT) prompting has significantly improved the reasoning capabilities of large language models (LLMs). However, conventional CoT often relies on unstructured, flat reasoning chains that suffer from redundancy and suboptimal performance. In this work, we introduce Hierarchical Chain-of-Thought (Hi-CoT), a structured reasoning paradigm specifically designed to address the challenges of complex, multi-step reasoning. Hi-CoT decomposes the reasoning process into hierarchical substeps by alternating between instructional planning and step-by-step execution. This decomposition enables LLMs to better manage long reasoning horizons and maintain logical coherence. Extensive evaluations across diverse LLMs and mathematical reasoning benchmarks show that Hi-CoT consistently improves average accuracy by 6.2% (up to 61.4% on certain models and tasks) while reducing reasoning trace length by 13.9% compared to CoT. We further show that accuracy and efficiency are maximized when models strictly adhere to the hierarchical structure. Our code is available at https://github.com/XingshuaiHuang/Hi-CoT.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 32 references that are in the map. First posted on arXiv 31 Mar 2026.
Builds on
20 of 32 references.
Closest ancestors · 6
Same direction, newest first
- 2025Enhancing Long Chain-of-Thought Reasoning through Multi-Path Plan Aggregation13 Oct
- 2025Revisiting Chain-of-Thought Prompting: Zero-shot Can Be Stronger than Few-shot17 Jun
- 2025Chain of Draft: Thinking Faster by Writing Less25 Feb
- 2022Automatic Chain of Thought Prompting in Large Language Models7 Oct
- 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Models21 May
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
Other references · 14
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2025Qwen3 Technical Reportcited by 8,820
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
- 2021CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Taskscited by 200
- 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMscited by 127
- 2025ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learningcited by 79
- 2025ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templatescited by 63
- 2025Emergent Hierarchical Reasoning in LLMs through Reinforcement Learningcited by 42
- 2024In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thoughtcited by 26
- 2025MetaScale: Test-Time Scaling with Evolving Meta-Thoughtscited by 13
- 2025Memory-Augmented Transformers: A Systematic Review from Neuroscience Principles to Enhanced Model Architecturescited by 9
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
- 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
- 2024Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs
- 2024Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs
- 2025Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
- 2024Understanding Chain-of-Thought in LLMs through Information Theory
- 2025Focused Chain-of-Thought: Efficient LLM Reasoning via Structured Input Information
- 2025Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning