Multi-hop Question Answering
Not on the map. Why?
Abstract (arXiv)
The task of Question Answering (QA) has attracted significant research interest for long. Its relevance to language understanding and knowledge retrieval tasks, along with the simple setting makes the task of QA crucial for strong AI systems. Recent success on simple QA tasks has shifted the focus to more complex settings. Among these, Multi-Hop QA (MHQA) is one of the most researched tasks over the recent years. In broad terms, MHQA is the task of answering natural language questions that involve extracting and combining multiple pieces of information and doing multiple steps of reasoning. An example of a multi-hop question would be "The Argentine PGA Championship record holder has won how many tournaments worldwide?". Answering the question would need two pieces of information: "Who is the record holder for Argentine PGA Championship tournaments?" and "How many tournaments did [Answer of Sub Q1] win?". The ability to answer multi-hop questions and perform multi step reasoning can significantly improve the utility of NLP systems. Consequently, the field has seen a surge with high quality datasets, models and evaluation strategies. The notion of 'multiple hops' is somewhat abstract which results in a large variety of tasks that require multi-hop reasoning. This leads to different datasets and models that differ significantly from each other and makes the field challenging to generalize and survey. We aim to provide a general and formal definition of the MHQA task, and organize and summarize existing MHQA frameworks. We also outline some best practices for building MHQA datasets. This book provides a systematic and thorough introduction as well as the structuring of the existing attempts to this highly interesting, yet quite challenging task.
Key facts. Cited by 64 papers in the map (data through 14 Oct 2026). First posted on arXiv 19 Apr 2022.
Builds on
0 of 0 references.
No references here yet.
Built on by
Citing papers per year: 2022: 3; 2023: 12; 2024: 15; 2025: 19; 2026: 15.
64 papers.
Sort:
By area:
- 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
- 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 326
- 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 213
- 2023Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answeringcited by 119
- 2023Why Does ChatGPT Fall Short in Providing Truthful Answers?cited by 54
- 2024Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approachcited by 40
- 2024Multi-LogiEval: Towards Evaluating Multi-Step Logical Reasoning Ability of Large Language Modelscited by 39
- 2023Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Modelscited by 37
- 2023WaterBench: Towards Holistic Evaluation of Watermarks for Large Language Modelscited by 32
- 2024TurkishMMLU: Measuring Massive Multitask Language Understanding in Turkishcited by 32
- 2023Self-prompted Chain-of-Thought on Large Language Models for Open-domain Multi-hop Reasoningcited by 21
- 2024REAPER: Reasoning based Retrieval Planning for Complex RAG Systemscited by 16
38 more
- 2024Lost in the Middle, and In-Between: Enhancing Language Models' Ability to Reason Over Long Contexts in Multi-Hop QAcited by 13
- 2024Leveraging Logical Rules in Knowledge Editing: A Cherry on the Topcited by 12
- 2025Credible Plan-Driven RAG Method for Multi-Hop Question Answeringcited by 12
- 2024OmniQuery: Contextually Augmenting Captured Multimodal Memory to Enable Personal Question Answeringcited by 10
- 2026DeepResearch-9K: A Challenging Benchmark Dataset of Deep-Research Agentcited by 10
- 2025A Fano-Style Accuracy Upper Bound for LLM Single-Pass Reasoning in Multi-Hop QAcited by 9
- 2024RAG-based Question Answering over Heterogeneous Data and Textcited by 7
- 2026GRACE: Reinforcement Learning for Grounded Response and Abstention under Contextual Evidencecited by 7
- 2023EpiK-Eval: Evaluation for Language Models as Epistemic Modelscited by 6
- 2023Disentangling Extraction and Reasoning in Multi-hop Spatial Reasoningcited by 6
- 2025Self-Evaluating LLMs for Multi-Step Tasks: Stepwise Confidence Estimation for Failure Detectioncited by 6
- 2025Wrong Answers Can Also Be Useful: PlausibleQA -- A Large-Scale QA Dataset with Answer Plausibility Scorescited by 5
- 2025On Generalization across Measurement Systems: LLMs Entail More Test-Time Compute for Underrepresented Culturescited by 5
- 2025Safeguarding RAG Pipelines with GMTP: A Gradient-based Masked Token Probability Method for Poisoned Document Detectioncited by 5
- 2025Who's Asking? Evaluating LLM Robustness to Inquiry Personas in Factual Question Answeringcited by 5
- 2022Rethinking Label Smoothing on Multi-hop Question Answeringcited by 4
- 2023Performance Prediction for Multi-hop Questionscited by 4
- 2024Dr3: Ask Large Language Models Not to Give Off-Topic Answers in Open Domain Multi-Hop Question Answeringcited by 4
- 2024WikiHint: A Human-Annotated Dataset for Hint Ranking and Generationcited by 4
- 2025EpidemIQs: Prompt-to-Paper LLM Agents for Epidemic Modeling and Analysiscited by 4
- 2022Reasoning Circuits: Few-shot Multihop Question Generation with Structured Rationalescited by 3
- 2025Evaluating Multi-Hop Reasoning in Large Language Models: A Chemistry-Centric Case Studycited by 3
- 2025BELLE: A Bi-Level Multi-Agent Reasoning Framework for Multi-Hop Question Answeringcited by 3
- 2025MDBench: A Synthetic Multi-Document Reasoning Benchmark Generated with Knowledge Guidancecited by 3
- 2026S3-R1: Learning to Retrieve and Answer Step-by-Step with Synthetic Datacited by 3
- 2023Furthest Reasoning with Plan Assessment: Stable Reasoning Path with Retrieval-Augmented Large Language Modelscited by 2
- 2023Uncertainty Guided Global Memory Improves Multi-Hop Question Answeringcited by 2
- 2025QuOTE: Question-Oriented Text Embeddingscited by 2
- 2025Bridging Dual Knowledge Graphs for Multi-Hop Question Answering in Construction Safetycited by 2
- 2024AS-ES Learning: Towards Efficient CoT Learning in Small Modelscited by 1
- 2024Measuring text summarization factuality using atomic facts entailment metrics in the context of retrieval augmented generationcited by 1
- 2024Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledgecited by 1
- 2024Review-Then-Refine: A Dynamic Framework for Multi-Hop Question Answering with Temporal Adaptabilitycited by 1
- 2025CaresAI at BioCreative IX Track 1 -- LLM for Biomedical QAcited by 1
- 2025Equipping Retrieval-Augmented Large Language Models with Document Structure Awarenesscited by 1
- 2025MGA-VQA: Secure and Interpretable Graph-Augmented Visual Question Answering with Memory-Guided Protection Against Unauthorized Knowledge Usecited by 1
- 2025Leveraging Spreading Activation for Improved Document Retrieval in Knowledge-Graph-Based RAG Systemscited by 1
- 2026Two Pathways to Truthfulness: On the Intrinsic Encoding of LLM Hallucinationscited by 1
50 most cited of 64 All 64 (JSON)
- 2026ConRAG: Lightweight inference of multi-hop relations28 Sep 2026
- 2026XHotpotQA: A Benchmark for Cross-Lingual Knowledge Composition in Multi-Hop Question Answering23 Aug 2026
- 2026SAG: SQL-Retrieval Augmented Generation with Query-Time Dynamic Hyperedges12 Aug 2026
- 2026SAG: SQL-Retrieval Augmented Generation with Query-Time Dynamic Hyperedges14 Jun 2026
- 2026Trading Complexity for Expressivity Through Structured Generalized Linear Token Mixing29 May 2026
- 2026S3-R1: Learning to Retrieve and Answer Step-by-Step with Synthetic Data2 May 2026
- 2026A Reproducibility Study of Metacognitive Retrieval-Augmented Generation21 Apr 2026
- 2026Same Outcomes, Different Journeys: A Trace-Level Framework for Comparing Human and GUI-Agent Behavior in Production Search Systems9 Apr 2026
- 2026Structured Linked Data as a Memory Layer for Agent-Orchestrated Retrieval11 Mar 2026
- 2026DeepResearch-9K: A Challenging Benchmark Dataset of Deep-Research Agent1 Mar 2026
- 2026Evaluating Prompt Engineering Techniques for RAG in Small Language Models: A Multi-Hop QA Approach14 Feb 2026
- 2026No Shortcuts to Culture: Indonesian Multi-hop Question Answering for Complex Cultural Understanding3 Feb 2026
38 more
- 2026User-Centric Evidence Ranking for Attribution and Fact Verification29 Jan 2026
- 2026Two Pathways to Truthfulness: On the Intrinsic Encoding of LLM Hallucinations12 Jan 2026
- 2026GRACE: Reinforcement Learning for Grounded Response and Abstention under Contextual Evidence8 Jan 2026
- 2025Leveraging Spreading Activation for Improved Document Retrieval in Knowledge-Graph-Based RAG Systems17 Dec 2025
- 2025MGA-VQA: Secure and Interpretable Graph-Augmented Visual Question Answering with Memory-Guided Protection Against Unauthorized Knowledge Use22 Nov 2025
- 2025Self-Evaluating LLMs for Multi-Step Tasks: Stepwise Confidence Estimation for Failure Detection10 Nov 2025
- 2025Who's Asking? Evaluating LLM Robustness to Inquiry Personas in Factual Question Answering14 Oct 2025
- 2025Equipping Retrieval-Augmented Large Language Models with Document Structure Awareness5 Oct 2025
- 2025A Fano-Style Accuracy Upper Bound for LLM Single-Pass Reasoning in Multi-Hop QA25 Sep 2025
- 2025EpidemIQs: Prompt-to-Paper LLM Agents for Epidemic Modeling and Analysis24 Sep 2025
- 2025CaresAI at BioCreative IX Track 1 -- LLM for Biomedical QA31 Aug 2025
- 2025Safeguarding RAG Pipelines with GMTP: A Gradient-based Masked Token Probability Method for Poisoned Document Detection24 Jul 2025
- 2025Bridging Dual Knowledge Graphs for Multi-Hop Question Answering in Construction Safety18 Jul 2025
- 2025MDBench: A Synthetic Multi-Document Reasoning Benchmark Generated with Knowledge Guidance17 Jun 2025
- 2025NeuroFaith: Evaluating LLM Self-Explanation Faithfulness via Internal Representation Alignment10 Jun 2025
- 2025On Generalization across Measurement Systems: LLMs Entail More Test-Time Compute for Underrepresented Cultures3 Jun 2025
- 2025KARE-RAG: Knowledge-Aware Refinement and Enhancement for RAG3 Jun 2025
- 2025BELLE: A Bi-Level Multi-Agent Reasoning Framework for Multi-Hop Question Answering17 May 2025
- 2025Credible Plan-Driven RAG Method for Multi-Hop Question Answering23 Apr 2025
- 2025Evaluating Multi-Hop Reasoning in Large Language Models: A Chemistry-Centric Case Study23 Apr 2025
- 2025Wrong Answers Can Also Be Useful: PlausibleQA -- A Large-Scale QA Dataset with Answer Plausibility Scores22 Feb 2025
- 2025QuOTE: Question-Oriented Text Embeddings16 Feb 2025
- 2024Review-Then-Refine: A Dynamic Framework for Multi-Hop Question Answering with Temporal Adaptability19 Dec 2024
- 2024Lost in the Middle, and In-Between: Enhancing Language Models' Ability to Reason Over Long Contexts in Multi-Hop QA13 Dec 2024
- 2024Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge11 Dec 2024
- 2024RAG-based Question Answering over Heterogeneous Data and Text10 Dec 2024
- 2024WikiHint: A Human-Annotated Dataset for Hint Ranking and Generation2 Dec 2024
- 2024OmniQuery: Contextually Augmenting Captured Multimodal Memory to Enable Personal Question Answering12 Sep 2024
- 2024Measuring text summarization factuality using atomic facts entailment metrics in the context of retrieval augmented generation27 Aug 2024
- 2024REAPER: Reasoning based Retrieval Planning for Complex RAG Systems26 Jul 2024
- 2024Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approach18 Jul 2024
- 2024TurkishMMLU: Measuring Massive Multitask Language Understanding in Turkish17 Jul 2024
- 2024Multi-LogiEval: Towards Evaluating Multi-Step Logical Reasoning Ability of Large Language Models24 Jun 2024
- 2024Leveraging Logical Rules in Knowledge Editing: A Cherry on the Top24 May 2024
- 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models29 Apr 2024
- 2024Dr3: Ask Large Language Models Not to Give Off-Topic Answers in Open Domain Multi-Hop Question Answering19 Mar 2024
- 2024AS-ES Learning: Towards Efficient CoT Learning in Small Models4 Mar 2024
- 2023Uncertainty Guided Global Memory Improves Multi-Hop Question Answering29 Nov 2023
50 newest of 64 All 64 (JSON)
- 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
- 2024Multi-LogiEval: Towards Evaluating Multi-Step Logical Reasoning Ability of Large Language Modelscited by 39
- 2023Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Modelscited by 37
- 2023Self-prompted Chain-of-Thought on Large Language Models for Open-domain Multi-hop Reasoningcited by 21
- 2024REAPER: Reasoning based Retrieval Planning for Complex RAG Systemscited by 16
- 2024Lost in the Middle, and In-Between: Enhancing Language Models' Ability to Reason Over Long Contexts in Multi-Hop QAcited by 13
- 2025A Fano-Style Accuracy Upper Bound for LLM Single-Pass Reasoning in Multi-Hop QAcited by 9
- 2023Disentangling Extraction and Reasoning in Multi-hop Spatial Reasoningcited by 6
- 2025On Generalization across Measurement Systems: LLMs Entail More Test-Time Compute for Underrepresented Culturescited by 5
- 2022Reasoning Circuits: Few-shot Multihop Question Generation with Structured Rationalescited by 3
- 2025Evaluating Multi-Hop Reasoning in Large Language Models: A Chemistry-Centric Case Studycited by 3
- 2025BELLE: A Bi-Level Multi-Agent Reasoning Framework for Multi-Hop Question Answeringcited by 3
12 most cited of 18 All 18 (JSON)
- 2026SAG: SQL-Retrieval Augmented Generation with Query-Time Dynamic Hyperedges12 Aug 2026
- 2026SAG: SQL-Retrieval Augmented Generation with Query-Time Dynamic Hyperedges14 Jun 2026
- 2025A Fano-Style Accuracy Upper Bound for LLM Single-Pass Reasoning in Multi-Hop QA25 Sep 2025
- 2025MDBench: A Synthetic Multi-Document Reasoning Benchmark Generated with Knowledge Guidance17 Jun 2025
- 2025On Generalization across Measurement Systems: LLMs Entail More Test-Time Compute for Underrepresented Cultures3 Jun 2025
- 2025BELLE: A Bi-Level Multi-Agent Reasoning Framework for Multi-Hop Question Answering17 May 2025
- 2025Evaluating Multi-Hop Reasoning in Large Language Models: A Chemistry-Centric Case Study23 Apr 2025
- 2024Lost in the Middle, and In-Between: Enhancing Language Models' Ability to Reason Over Long Contexts in Multi-Hop QA13 Dec 2024
- 2024Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge11 Dec 2024
- 2024REAPER: Reasoning based Retrieval Planning for Complex RAG Systems26 Jul 2024
- 2024Multi-LogiEval: Towards Evaluating Multi-Step Logical Reasoning Ability of Large Language Models24 Jun 2024
- 2024AS-ES Learning: Towards Efficient CoT Learning in Small Models4 Mar 2024
12 newest of 18 All 18 (JSON)
- 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 213
- 2024Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approachcited by 40
- 2025Credible Plan-Driven RAG Method for Multi-Hop Question Answeringcited by 12
- 2026GRACE: Reinforcement Learning for Grounded Response and Abstention under Contextual Evidencecited by 7
- 2025Safeguarding RAG Pipelines with GMTP: A Gradient-based Masked Token Probability Method for Poisoned Document Detectioncited by 5
- 2025QuOTE: Question-Oriented Text Embeddingscited by 2
- 2024Measuring text summarization factuality using atomic facts entailment metrics in the context of retrieval augmented generationcited by 1
- 2025Equipping Retrieval-Augmented Large Language Models with Document Structure Awarenesscited by 1
- 2025Leveraging Spreading Activation for Improved Document Retrieval in Knowledge-Graph-Based RAG Systemscited by 1
- 2025KARE-RAG: Knowledge-Aware Refinement and Enhancement for RAGcited by 0
- 2026Evaluating Prompt Engineering Techniques for RAG in Small Language Models: A Multi-Hop QA Approachcited by 0
- 2026Structured Linked Data as a Memory Layer for Agent-Orchestrated Retrievalcited by 0
12 most cited of 13 All 13 (JSON)
- 2026A Reproducibility Study of Metacognitive Retrieval-Augmented Generation21 Apr 2026
- 2026Structured Linked Data as a Memory Layer for Agent-Orchestrated Retrieval11 Mar 2026
- 2026Evaluating Prompt Engineering Techniques for RAG in Small Language Models: A Multi-Hop QA Approach14 Feb 2026
- 2026GRACE: Reinforcement Learning for Grounded Response and Abstention under Contextual Evidence8 Jan 2026
- 2025Leveraging Spreading Activation for Improved Document Retrieval in Knowledge-Graph-Based RAG Systems17 Dec 2025
- 2025Equipping Retrieval-Augmented Large Language Models with Document Structure Awareness5 Oct 2025
- 2025Safeguarding RAG Pipelines with GMTP: A Gradient-based Masked Token Probability Method for Poisoned Document Detection24 Jul 2025
- 2025KARE-RAG: Knowledge-Aware Refinement and Enhancement for RAG3 Jun 2025
- 2025Credible Plan-Driven RAG Method for Multi-Hop Question Answering23 Apr 2025
- 2025QuOTE: Question-Oriented Text Embeddings16 Feb 2025
- 2024Measuring text summarization factuality using atomic facts entailment metrics in the context of retrieval augmented generation27 Aug 2024
- 2024Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approach18 Jul 2024
12 newest of 13 All 13 (JSON)
- 2026DeepResearch-9K: A Challenging Benchmark Dataset of Deep-Research Agentcited by 10
- 2025EpidemIQs: Prompt-to-Paper LLM Agents for Epidemic Modeling and Analysiscited by 4
- 2026S3-R1: Learning to Retrieve and Answer Step-by-Step with Synthetic Datacited by 3
- 2026Same Outcomes, Different Journeys: A Trace-Level Framework for Comparing Human and GUI-Agent Behavior in Production Search Systemscited by 0
- 2026S3-R1: Learning to Retrieve and Answer Step-by-Step with Synthetic Data2 May 2026
- 2026Same Outcomes, Different Journeys: A Trace-Level Framework for Comparing Human and GUI-Agent Behavior in Production Search Systems9 Apr 2026
- 2026DeepResearch-9K: A Challenging Benchmark Dataset of Deep-Research Agent1 Mar 2026
- 2025EpidemIQs: Prompt-to-Paper LLM Agents for Epidemic Modeling and Analysis24 Sep 2025
- 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 326
- 2023Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answeringcited by 119
- 2023Why Does ChatGPT Fall Short in Providing Truthful Answers?cited by 54
- 2024TurkishMMLU: Measuring Massive Multitask Language Understanding in Turkishcited by 32
- 2024Leveraging Logical Rules in Knowledge Editing: A Cherry on the Topcited by 12
- 2024OmniQuery: Contextually Augmenting Captured Multimodal Memory to Enable Personal Question Answeringcited by 10
- 2024RAG-based Question Answering over Heterogeneous Data and Textcited by 7
- 2023EpiK-Eval: Evaluation for Language Models as Epistemic Modelscited by 6
- 2025Self-Evaluating LLMs for Multi-Step Tasks: Stepwise Confidence Estimation for Failure Detectioncited by 6
- 2025Wrong Answers Can Also Be Useful: PlausibleQA -- A Large-Scale QA Dataset with Answer Plausibility Scorescited by 5
- 2025Who's Asking? Evaluating LLM Robustness to Inquiry Personas in Factual Question Answeringcited by 5
- 2022Rethinking Label Smoothing on Multi-hop Question Answeringcited by 4
12 most cited of 28 All 28 (JSON)
- 2026ConRAG: Lightweight inference of multi-hop relations28 Sep 2026
- 2026XHotpotQA: A Benchmark for Cross-Lingual Knowledge Composition in Multi-Hop Question Answering23 Aug 2026
- 2026Trading Complexity for Expressivity Through Structured Generalized Linear Token Mixing29 May 2026
- 2026No Shortcuts to Culture: Indonesian Multi-hop Question Answering for Complex Cultural Understanding3 Feb 2026
- 2026User-Centric Evidence Ranking for Attribution and Fact Verification29 Jan 2026
- 2026Two Pathways to Truthfulness: On the Intrinsic Encoding of LLM Hallucinations12 Jan 2026
- 2025MGA-VQA: Secure and Interpretable Graph-Augmented Visual Question Answering with Memory-Guided Protection Against Unauthorized Knowledge Use22 Nov 2025
- 2025Self-Evaluating LLMs for Multi-Step Tasks: Stepwise Confidence Estimation for Failure Detection10 Nov 2025
- 2025Who's Asking? Evaluating LLM Robustness to Inquiry Personas in Factual Question Answering14 Oct 2025
- 2025CaresAI at BioCreative IX Track 1 -- LLM for Biomedical QA31 Aug 2025
- 2025Bridging Dual Knowledge Graphs for Multi-Hop Question Answering in Construction Safety18 Jul 2025
- 2025NeuroFaith: Evaluating LLM Self-Explanation Faithfulness via Internal Representation Alignment10 Jun 2025
12 newest of 28 All 28 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2022Semantic Sentence Composition Reasoning for Multi-Hop Question Answering
- 2022From Easy to Hard: Two-stage Selector and Reader for Multi-hop Question Answering
- 2023Analyzing the Effectiveness of the Underlying Reasoning Tasks in Multi-hop Question Answering
- 2026Overview of the MedHopQA track at BioCreative IX: track description, participation and evaluation of systems for multi-hop medical question answering
- 2024MoreHopQA: More Than Multi-hop Reasoning
- 2023Answering Questions by Meta-Reasoning over Multiple Chains of Thought
- 2022Better Retrieval May Not Lead to Better Question Answering
- 2024MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledge