mlsanityData to 14 Oct 2026

Few-Shot Data Synthesis for Open Domain Multi-Hop Question Answering

arXiv 2305.13691 ↗First posted 23 May 2023cs.CLcited by 9

On the map: Synthetic data and self-improvement for LLMs → Synthetic reasoning data and bootstrapping

Abstract (arXiv)

Few-shot learning for open domain multi-hop question answering typically relies on the incontext learning capability of large language models (LLMs). While powerful, these LLMs usually contain tens or hundreds of billions of parameters, making them rather inefficient at inference time. To improve performance of smaller language models, we propose a data synthesis framework for multi-hop question answering that requires less than 10 human annotated question answer pairs. Our framework depends only on rich, naturally-occurring relationships among documents and is built upon the data generation functions parameterized by LLMs and prompts. We synthesize millions of multi-hop questions and claims to finetune language models, evaluated on popular benchmarks for multi-hop question answering and fact verification. Empirically, our approach improves model performance significantly, allowing the finetuned models to be competitive with GPT-3.5 based approaches while being almost one-third the size in parameter count.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Synthetic reasoning data and bootstrapping, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 44 of its 66 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

44 of 66 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Large Language Models Can Self-Improve20 Oct

Other references · 43

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  8. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  10. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  11. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  12. 2017Billion-scale similarity search with GPUscited by 2,497
31 more
  1. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  2. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  3. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  4. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  5. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  6. 2016Sequence-Level Knowledge Distillationcited by 801
  7. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  8. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  9. 2013Do Deep Nets Really Need to be Deep?cited by 474
  10. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 397
  11. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 318
  12. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  13. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 211
  14. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 200
  15. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  16. 2021Want To Reduce Labeling Cost? GPT-3 Can Helpcited by 196
  17. 2021Generating Datasets with Pretrained Language Modelscited by 193
  18. 2022Improving Passage Retrieval with Zero-Shot Question Generationcited by 165
  19. 2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
  20. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 118
  21. 2023SCOTT: Self-Consistent Chain-of-Thought Distillationcited by 105
  22. 2022Successive Prompting for Decomposing Complex Questionscited by 94
  23. 2023Answering Questions by Meta-Reasoning over Multiple Chains of Thoughtcited by 87
  24. 2019Answering Complex Open-domain Questions Through Iterative Query Generationcited by 60
  25. 2020Semantic Graphs for Generating Deep Questionscited by 38
  26. 2020Unsupervised Multi-hop Question Answering by Question Generationcited by 38
  27. 2022QAmeleon: Multilingual QA with Only 5 Examplescited by 25
  28. 2020Multi-hop Question Generation with Graph Convolutional Networkcited by 20
  29. 2021Question Answering Infused Pre-training of General-Purpose Contextualized Representationscited by 19
  30. 2023ZARA: Improving Few-Shot Self-Rationalization for Small Language Modelscited by 8
  31. 2020Stronger Transformers for Neural Multi-Hop Question Generationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 2; 2026: 1.

9 papers.

  1. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Casescited by 393
  2. 2024DataGen: Unified Synthetic Dataset Generation via Large Language Modelscited by 23
  3. 2023Semi-Structured Chain-of-Thought: Integrating Multiple Sources of Knowledge for Improved Language Model Reasoningcited by 9
  4. 2024RAGProbe: An Automated Approach for Evaluating RAG Applicationscited by 7
  5. 2024FM2DS: Few-Shot Multimodal Multihop Data Synthesis with Knowledge Distillation for Question Answeringcited by 7
  6. 2025ImpRAG: Retrieval-Augmented Generation with Implicit Queriescited by 5
  7. 2025LimRank: Less is More for Reasoning-Intensive Information Rerankingcited by 3
  8. 2026RAGExplorer: A Visual Analytics System for the Comparative Diagnosis of RAG Systemscited by 2
  9. 2024Rehearsing Answers to Probable Questions with Perspective-Takingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Zero-Shot Multi-Hop Question Answering via Monte-Carlo Tree Search with Large Language Models
  2. 2023Multimodal Multi-Hop Question Answering Through a Conversation Between Tools and Efficiently Finetuned Large Language Models
  3. 2024FSM: A Finite State Machine Based Zero-Shot Prompting Paradigm for Multi-Hop Question Answering
  4. 2024SG-FSM: A Self-Guiding Zero-Shot Prompting Paradigm for Multi-Hop Question Answering Based on Finite State Machine
  5. 2026Fine-Tuning vs. RAG for Multi-Hop Question Answering with Novel Knowledge
  6. 2024Prompting-based Synthetic Data Generation for Few-Shot Question Answering
  7. 2022Understanding and Improving Zero-shot Multi-hop Reasoning in Generative Question Answering
  8. 2022Internet-augmented language models through few-shot prompting for open-domain question answering