MARCH: Evaluating the Intersection of Ambiguity Interpretation and Multi-hop Inference
On the map: LLM reasoning → Knowledge- and tool-augmented reasoning
Abstract (arXiv)
Real-world multi-hop QA is naturally linked with ambiguity, where a single query can trigger multiple reasoning paths that require independent resolution. Since ambiguity can occur at any stage, models must navigate layered uncertainty throughout the entire reasoning chain. Despite its prevalence in real-world user queries, previous benchmarks have primarily focused on single-hop ambiguity, leaving the complex interaction between multi-step inference and layered ambiguity underexplored. In this paper, we introduce MARCH, a benchmark for their intersection, with 2,209 multi-hop ambiguous questions curated via multi-LLM verification and validated by human annotation with strong agreement. Our experiments reveal that even state-of-the-art models struggle with MARCH, confirming that combining ambiguity resolution with multi-step reasoning is a significant challenge. To address this, we propose CLARION, a two-stage agentic framework that explicitly decouples ambiguity planning from evidence-driven reasoning, significantly outperforms existing approaches, and paves the way for robust reasoning systems.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 25 references that are in the map. First posted on arXiv 26 Sep 2025.
Builds on
17 of 25 references.
References · 17
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
- 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
- 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 461
- 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 390
- 2024MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queriescited by 257
- 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 234
- 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
- 2023Clarify When Necessary: Resolving Ambiguity Through Interaction with LMscited by 98
- 2024Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questionscited by 76
5 more
- 2024AmbigDocs: Reasoning across Documents on Different Entities under the Same Namecited by 21
- 2024MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledgecited by 20
- 2025CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answeringcited by 15
- 2024Diversify-verify-adapt: Efficient and Robust Retrieval-Augmented Ambiguous Question Answeringcited by 10
- 2025Clarifying Ambiguities: on the Role of Ambiguity Types in Prompting Methods for Clarification Generationcited by 10
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026HopRefusalBench: Diagnosing Refusal Failures in Search-Augmented Agents for Multi-Hop Reasoning
- 2025Hop, Skip, and Overthink: Diagnosing Why Reasoning Models Fumble during Multi-Hop Analysis
- 2026Retrieval--Reasoning Processes for Multi-hop Question Answering: A Four-Axis Design Framework and Empirical Trends
- 2026Locally Sound, Globally Insufficient: The Local-Global Gap in Multi-Hop Reasoning
- 2026Diversity is Not Ambiguity: Toward Accurate and Efficient Ambiguity Detection for Open-Domain QA
- 2025MultiHoax: A Dataset of Multi-hop False-Premise Questions
- 2026From Answers to Interpretations: Rethinking Ambiguity-Induced Aleatoric Uncertainty Estimation in LLMs
- 2026SAFE: An LLM-as-Verifier Framework for Evidence-Grounded Multi-Hop Reasoning