# Hierarchical Deconstruction of LLM Reasoning: A Graph-Based Framework for Analyzing Knowledge Utilization

arXiv: https://arxiv.org/abs/2406.19502 · HTML: https://mlsanity.com/paper/2406.19502/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 0 of its 62 references that are in the map.
- First posted on arXiv 27 Jun 2024.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Built on by (most cited first, 9 of 9)

- [Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?](https://mlsanity.com/paper/2411.16679/) (2024)
- [LLM-as-an-Interviewer: Beyond Static Testing Through Dynamic LLM Evaluation](https://mlsanity.com/paper/2412.10424/) (2024)
- [LogicQA: Logical Anomaly Detection with Vision Language Model Generated Questions](https://mlsanity.com/paper/2503.20252/) (2025)
- [Grounding LLM Reasoning with Knowledge Graphs](https://mlsanity.com/paper/2502.13247/) (2025)
- [Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning](https://mlsanity.com/paper/2506.04723/) (2025)
- [REMA: A Unified Reasoning Manifold Framework for Interpreting Large Language Model](https://mlsanity.com/paper/2509.22518/) (2025)
- [Can LLMs Estimate Cognitive Complexity of Reading Comprehension Items?](https://mlsanity.com/paper/2510.25064/) (2025)
- [JudgeAgent: Beyond Static Benchmarks for Knowledge-Driven and Dynamic LLM Evaluation](https://mlsanity.com/paper/2509.02097/) (2025)
- [LLMs Do Not Grade Essays Like Humans](https://mlsanity.com/paper/2603.23714/) (2026)

## Abstract (arXiv)

Despite the advances in large language models (LLMs), how they use their knowledge for reasoning is not yet well understood. In this study, we propose a method that deconstructs complex real-world questions into a graph, representing each question as a node with predecessors of background knowledge needed to solve the question. We develop the DepthQA dataset, deconstructing questions into three depths: (i) recalling conceptual knowledge, (ii) applying procedural knowledge, and (iii) analyzing strategic knowledge. Based on a hierarchical graph, we quantify forward discrepancy, a discrepancy in LLM performance on simpler sub-problems versus complex questions. We also measure backward discrepancy where LLMs answer complex questions but struggle with simpler ones. Our analysis shows that smaller models exhibit more discrepancies than larger models. Distinct patterns of discrepancies are observed across model capacity and possibility of training data memorization. Additionally, guiding models from simpler to complex questions through multi-turn interactions improves performance across model sizes, highlighting the importance of structured intermediate steps in knowledge reasoning. This work enhances our understanding of LLM reasoning and suggests ways to improve their problem-solving abilities.