# Improving Question Answering with External Knowledge

arXiv: https://arxiv.org/abs/1902.00993 · HTML: https://mlsanity.com/paper/1902.00993/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Feb 2019.

## Built on by (most cited first, 16 of 16)

- [QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering](https://mlsanity.com/paper/2104.06378/) (2021)
- [Aligning Language Models to User Opinions](https://mlsanity.com/paper/2305.14929/) (2023)
- [UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training](https://mlsanity.com/paper/2202.12359/) (2022)
- [PRCA: Fitting Black-Box Large Language Models for Retrieval Question Answering via Pluggable Reward-Driven Contextual Adapter](https://mlsanity.com/paper/2310.18347/) (2023)
- [Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models](https://mlsanity.com/paper/2210.16433/) (2022)
- [Joint Biomedical Entity and Relation Extraction with Knowledge-Enhanced Collective Inference](https://mlsanity.com/paper/2105.13456/) (2021)
- [Does External Knowledge Help Explainable Natural Language Inference? Automatic Evaluation vs. Human Ratings](https://mlsanity.com/paper/2109.07833/) (2021)
- [Long-Tailed Question Answering in an Open World](https://mlsanity.com/paper/2305.06557/) (2023)
- [Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Data](https://mlsanity.com/paper/2102.01226/) (2021)
- [OPERA: Harmonizing Task-Oriented Dialogs and Information Seeking Experience](https://mlsanity.com/paper/2206.12449/) (2022)
- [Using External Off-Policy Speech-To-Text Mappings in Contextual End-To-End Automated Speech Recognition](https://mlsanity.com/paper/2301.02736/) (2023)
- [An MRC Framework for Semantic Role Labeling](https://mlsanity.com/paper/2109.06660/) (2021)
- [Towards Faithful Knowledge Graph Explanation Through Deep Alignment in Commonsense Question Answering](https://mlsanity.com/paper/2310.04910/) (2023)
- [Memory-assisted prompt editing to improve GPT-3 after deployment](https://mlsanity.com/paper/2201.06009/) (2022)
- [ActKnow: Active External Knowledge Infusion Learning for Question Answering in Low Data Regime](https://mlsanity.com/paper/2112.09423/) (2021)
- [Recent Advances in Multi-Choice Machine Reading Comprehension: A Survey on Methods and Datasets](https://mlsanity.com/paper/2408.02114/) (2024)

## Abstract (arXiv)

We focus on multiple-choice question answering (QA) tasks in subject areas such as science, where we require both broad background knowledge and the facts from the given subject-area reference corpus. In this work, we explore simple yet effective methods for exploiting two sources of external knowledge for subject-area QA. The first enriches the original subject-area reference corpus with relevant text snippets extracted from an open-domain resource (i.e., Wikipedia) that cover potentially ambiguous concepts in the question and answer options. As in other QA research, the second method simply increases the amount of training data by appending additional in-domain subject-area instances. Experiments on three challenging multiple-choice science QA tasks (i.e., ARC-Easy, ARC-Challenge, and OpenBookQA) demonstrate the effectiveness of our methods: in comparison to the previous state-of-the-art, we obtain absolute gains in accuracy of up to 8.1%, 13.0%, and 12.8%, respectively. While we observe consistent gains when we introduce knowledge from Wikipedia, we find that employing additional QA training instances is not uniformly helpful: performance degrades when the added instances exhibit a higher level of difficulty than the original training data. As one of the first studies on exploiting unstructured external knowledge for subject-area QA, we hope our methods, observations, and discussion of the exposed limitations may shed light on further developments in the area.