# Leveraging Structured Information for Explainable Multi-hop Question Answering and Reasoning

arXiv: https://arxiv.org/abs/2311.03734 · HTML: https://mlsanity.com/paper/2311.03734/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 13 of its 35 references that are in the map.
- First posted on arXiv 7 Nov 2023.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 13 of 13)

- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [Decomposed Prompting: A Modular Approach for Solving Complex Tasks](https://mlsanity.com/paper/2210.02406/) (2022)
- [Faith and Fate: Limits of Transformers on Compositionality](https://mlsanity.com/paper/2305.18654/) (2023)
- [Faithful Chain-of-Thought Reasoning](https://mlsanity.com/paper/2301.13379/) (2023)
- [ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning](https://mlsanity.com/paper/2212.07919/) (2022)
- [WT5?! Training Text-to-Text Models to Explain their Predictions](https://mlsanity.com/paper/2004.14546/) (2020)
- [Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering](https://mlsanity.com/paper/1911.10470/) (2019)
- [Answering Questions by Meta-Reasoning over Multiple Chains of Thought](https://mlsanity.com/paper/2304.13007/) (2023)
- [Interpretable AMR-Based Question Decomposition for Multi-hop Question Answering](https://mlsanity.com/paper/2206.08486/) (2022)
- [Iterative Hierarchical Attention for Answering Complex Questions over Long Documents](https://mlsanity.com/paper/2106.00200/) (2021)
- [Multi-Paragraph Reasoning with Knowledge-enhanced Graph Neural Network](https://mlsanity.com/paper/1911.02170/) (2019)

## Built on by (most cited first, 22 of 22)

- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models](https://mlsanity.com/paper/2405.14831/) (2024)
- [Generate-then-Ground in Retrieval-Augmented Generation for Multi-hop Question Answering](https://mlsanity.com/paper/2406.14891/) (2024)
- [HOLMES: Hyper-Relational Knowledge Graphs for Multi-hop Question Answering using LLMs](https://mlsanity.com/paper/2406.06027/) (2024)
- [CAM: A Constructivist View of Agentic Memory for LLM-Based Reading Comprehension](https://mlsanity.com/paper/2510.05520/) (2025)
- [IQA-EVAL: Automatic Evaluation of Human-Model Interactive Question Answering](https://mlsanity.com/paper/2408.13545/) (2024)
- [PIKE-RAG: sPecIalized KnowledgE and Rationale Augmented Generation](https://mlsanity.com/paper/2501.11551/) (2025)
- [KiRAG: Knowledge-Driven Iterative Retriever for Enhancing Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.18397/) (2025)
- [Document-level Causal Relation Extraction with Knowledge-guided Binary Question Answering](https://mlsanity.com/paper/2410.04752/) (2024)
- [Cog-RAG: Cognitive-Inspired Dual-Hypergraph with Theme Alignment Retrieval-Augmented Generation](https://mlsanity.com/paper/2511.13201/) (2025)
- [KnowTrace: Bootstrapping Iterative Retrieval-Augmented Generation with Structured Knowledge Tracing](https://mlsanity.com/paper/2505.20245/) (2025)
- [Hybrid Graphs for Table-and-Text based Question Answering using LLMs](https://mlsanity.com/paper/2501.17767/) (2025)
- [Parse Trees Guided LLM Prompt Compression](https://mlsanity.com/paper/2409.15395/) (2024)
- [Information Re-Organization Improves Reasoning in Large Language Models](https://mlsanity.com/paper/2404.13985/) (2024)
- [GlobalRAG: Enhancing Global Reasoning in Multi-hop Question Answering via Reinforcement Learning](https://mlsanity.com/paper/2510.20548/) (2025)
- [DF-RAG: Query-Aware Diversity for Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.17212/) (2026)
- [Cross-Granularity Hypergraph Retrieval-Augmented Generation for Multi-hop Question Answering](https://mlsanity.com/paper/2508.11247/) (2025)
- [Only Ask What You Don't Know: Grounded Delta Planning for Efficient Multi-step RAG](https://mlsanity.com/paper/2606.22681/) (2026)
- [Pyramid-Driven Alignment: Pyramid Principle Guided Integration of Large Language Models and Knowledge Graphs](https://mlsanity.com/paper/2410.12298/) (2024)
- [STRIDE: Strategic Iterative Decision-Making for Retrieval-Augmented Multi-Hop Question Answering](https://mlsanity.com/paper/2604.17405/) (2026)
- [Resource-Friendly Dynamic Enhancement Chain for Multi-Hop Question Answering](https://mlsanity.com/paper/2506.17692/) (2025)
- [SSE-Bio: A Structured Self-Evolving Agent with Agentic Retrieval Policy for Multi-Hop Biomedical Reasoning](https://mlsanity.com/paper/2608.22132/) (2026)

## Abstract (arXiv)

Neural models, including large language models (LLMs), achieve superior performance on multi-hop question-answering. To elicit reasoning capabilities from LLMs, recent works propose using the chain-of-thought (CoT) mechanism to generate both the reasoning chain and the answer, which enhances the model's capabilities in conducting multi-hop reasoning. However, several challenges still remain: such as struggling with inaccurate reasoning, hallucinations, and lack of interpretability. On the other hand, information extraction (IE) identifies entities, relations, and events grounded to the text. The extracted structured information can be easily interpreted by humans and machines (Grishman, 2019). In this work, we investigate constructing and leveraging extracted semantic structures (graphs) for multi-hop question answering, especially the reasoning process. Empirical results and human evaluations show that our framework: generates more faithful reasoning chains and substantially improves the QA performance on two benchmark datasets. Moreover, the extracted structures themselves naturally provide grounded explanations that are preferred by humans, as compared to the generated reasoning chains and saliency-based explanations.