# HopWeaver: Cross-Document Synthesis of High-Quality and Authentic Multi-Hop Questions

arXiv: https://arxiv.org/abs/2505.15087 · HTML: https://mlsanity.com/paper/2505.15087/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 47 references that are in the map.
- First posted on arXiv 21 May 2025.

## Builds on (most cited first, 23 of 23)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge](https://mlsanity.com/paper/2410.02736/) (2024)
- [LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks](https://mlsanity.com/paper/2406.18403/) (2024)
- [Learning to Ask: Neural Question Generation for Reading Comprehension](https://mlsanity.com/paper/1705.00106/) (2017)
- [FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research](https://mlsanity.com/paper/2405.13576/) (2024)
- [Machine Learning for Synthetic Data Generation: A Review](https://mlsanity.com/paper/2302.04062/) (2023)
- [Compositional Questions Do Not Necessitate Multi-hop Reasoning](https://mlsanity.com/paper/1906.02900/) (2019)
- [Neural Question Generation from Text: A Preliminary Study](https://mlsanity.com/paper/1704.01792/) (2017)
- [Analyzing Dataset Annotation Quality Management in the Wild](https://mlsanity.com/paper/2307.08153/) (2023)
- [Guiding the Growth: Difficulty-Controllable Question Generation through Step-by-Step Rewriting](https://mlsanity.com/paper/2105.11698/) (2021)
- [Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources](https://mlsanity.com/paper/2409.08239/) (2024)
- [Evaluating the Consistency of LLM Evaluators](https://mlsanity.com/paper/2412.00543/) (2024)
- [MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledge](https://mlsanity.com/paper/2412.17032/) (2024)
- [LLM-Based Multi-Hop Question Answering with Knowledge Graph Integration in Evolving Environments](https://mlsanity.com/paper/2408.15903/) (2024)
- [QGEval: Benchmarking Multi-dimensional Evaluation for Question Generation](https://mlsanity.com/paper/2406.05707/) (2024)
- [Improving Question Generation with Multi-level Content Planning](https://mlsanity.com/paper/2310.13512/) (2023)
- [Synthetic Multimodal Question Generation](https://mlsanity.com/paper/2407.02233/) (2024)
- [Explainable Multi-hop Question Generation: An End-to-End Approach without Intermediate Question Labeling](https://mlsanity.com/paper/2404.00571/) (2024)
- [A Survey on Neural Question Generation: Methods, Applications, and Prospects](https://mlsanity.com/paper/2402.18267/) (2024)

## Built on by (most cited first, 2 of 2)

- [FORT-Searcher: Synthesizing Shortcut-Resistant Search Tasks for Training Deep Search Agents](https://mlsanity.com/paper/2606.12087/) (2026)
- [LimRank: Less is More for Reasoning-Intensive Information Reranking](https://mlsanity.com/paper/2510.23544/) (2025)

## Abstract (arXiv)

Multi-Hop Question Answering (MHQA) is crucial for evaluating the model's capability to integrate information from diverse sources. However, creating extensive and high-quality MHQA datasets is challenging: (i) manual annotation is expensive, and (ii) current synthesis methods often produce simplistic questions or require extensive manual guidance. This paper introduces HopWeaver, the first cross-document framework synthesizing authentic multi-hop questions without human intervention. HopWeaver synthesizes bridge and comparison questions through an innovative pipeline that identifies complementary documents and constructs authentic reasoning paths to ensure true multi-hop reasoning. We further present a comprehensive system for evaluating the synthesized multi-hop questions. Empirical evaluations demonstrate that the synthesized questions achieve comparable or superior quality to human-annotated datasets at a lower cost. Our framework provides a valuable tool for the research community: it can automatically generate challenging benchmarks from any raw corpus, which opens new avenues for both evaluation and targeted training to improve the reasoning capabilities of advanced question answering models, especially in domains with scarce resources.