# Prompt-based Conservation Learning for Multi-hop Question Answering

arXiv: https://arxiv.org/abs/2209.06923 · HTML: https://mlsanity.com/paper/2209.06923/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 34 references that are in the map.
- First posted on arXiv 14 Sep 2022.

## Builds on (most cited first, 29 of 29)

- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [MuSiQue: Multihop Questions via Single-hop Question Composition](https://mlsanity.com/paper/2108.00573/) (2021)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [The Web as a Knowledge-base for Answering Complex Questions](https://mlsanity.com/paper/1803.06643/) (2018)
- [PPT: Pre-trained Prompt Tuning for Few-shot Learning](https://mlsanity.com/paper/2109.04332/) (2021)
- [ERNIE 2.0: A Continual Pre-training Framework for Language Understanding](https://mlsanity.com/paper/1907.12412/) (2019)
- [Constructing Datasets for Multi-hop Reading Comprehension Across Documents](https://mlsanity.com/paper/1710.06481/) (2017)
- [LAMOL: LAnguage MOdeling for Lifelong Language Learning](https://mlsanity.com/paper/1909.03329/) (2019)
- [Multi-hop Reading Comprehension through Question Decomposition and Rescoring](https://mlsanity.com/paper/1906.02916/) (2019)
- [Unsupervised Question Decomposition for Question Answering](https://mlsanity.com/paper/2002.09758/) (2020)
- [Hierarchical Graph Network for Multi-hop Question Answering](https://mlsanity.com/paper/1911.03631/) (2019)
- [ELLE: Efficient Lifelong Pre-training for Emerging Data](https://mlsanity.com/paper/2203.06311/) (2022)
- [Dynamically Fused Graph Network for Multi-hop Reasoning](https://mlsanity.com/paper/1905.06933/) (2019)
- [Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QA](https://mlsanity.com/paper/1906.07132/) (2019)
- [Answering Complex Open-domain Questions Through Iterative Query Generation](https://mlsanity.com/paper/1910.07000/) (2019)
- [Select, Answer and Explain: Interpretable Multi-hop Reading Comprehension over Multiple Documents](https://mlsanity.com/paper/1911.00484/) (2019)
- [Answering Open-Domain Questions of Varying Reasoning Steps from Text](https://mlsanity.com/paper/2010.12527/) (2020)
- [Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction](https://mlsanity.com/paper/1905.08511/) (2019)
- [Do Multi-Hop Question Answering Systems Know How to Answer the Single-Hop Sub-Questions?](https://mlsanity.com/paper/2002.09919/) (2020)
- [Adaptive Information Seeking for Open-Domain Question Answering](https://mlsanity.com/paper/2109.06747/) (2021)
- [Lifelong Learning with Sketched Structural Regularization](https://mlsanity.com/paper/2104.08604/) (2021)
- [Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategy](https://mlsanity.com/paper/2107.11823/) (2021)

## Built on by (most cited first, 3 of 3)

- [Benchmarking Knowledge Boundary for Large Language Models: A Different Perspective on Model Evaluation](https://mlsanity.com/paper/2402.11493/) (2024)
- [GenDec: A robust generative Question-decomposition method for Multi-hop reasoning](https://mlsanity.com/paper/2402.11166/) (2024)
- [Prompting Explicit and Implicit Knowledge for Multi-hop Question Answering Based on Human Reading Process](https://mlsanity.com/paper/2402.19350/) (2024)

## Abstract (arXiv)

Multi-hop question answering (QA) requires reasoning over multiple documents to answer a complex question and provide interpretable supporting evidence. However, providing supporting evidence is not enough to demonstrate that a model has performed the desired reasoning to reach the correct answer. Most existing multi-hop QA methods fail to answer a large fraction of sub-questions, even if their parent questions are answered correctly. In this paper, we propose the Prompt-based Conservation Learning (PCL) framework for multi-hop QA, which acquires new knowledge from multi-hop QA tasks while conserving old knowledge learned on single-hop QA tasks, mitigating forgetting. Specifically, we first train a model on existing single-hop QA tasks, and then freeze this model and expand it by allocating additional sub-networks for the multi-hop QA task. Moreover, to condition pre-trained language models to stimulate the kind of reasoning required for specific multi-hop questions, we learn soft prompts for the novel sub-networks to perform type-specific reasoning. Experimental results on the HotpotQA benchmark show that PCL is competitive for multi-hop QA and retains good performance on the corresponding single-hop sub-questions, demonstrating the efficacy of PCL in mitigating knowledge loss by forgetting.