mlsanityData to 7 Oct 2026

Zero-Shot Open-Book Question Answering

arXiv 2111.11520 ↗First posted 22 Nov 2021cs.CL cs.IR cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Open book question answering is a subset of question answering tasks where the system aims to find answers in a given set of documents (open-book) and common knowledge about a topic. This article proposes a solution for answering natural language questions from a corpus of Amazon Web Services (AWS) technical documents with no domain-specific labeled data (zero-shot). These questions can have yes-no-none answers, short answers, long answers, or any combination of the above. This solution comprises a two-step architecture in which a retriever finds the right document and an extractor finds the answers in the retrieved document. We are introducing a new test dataset for open-book QA based on real customer questions on AWS technical documentation. After experimenting with several information retrieval systems and extractor models based on extractive language models, the solution attempts to find the yes-no-none answers and text answers in the same pass. The model is trained on the The Stanford Question Answering Dataset - SQuAD (Rajpurkaret al., 2016) and Natural Questions (Kwiatkowski et al., 2019) datasets. We were able to achieve 49% F1 and 39% exact match score (EM) end-to-end with no domain-specific training.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 19 of its 28 references that are in the map. First posted on arXiv 22 Nov 2021.

Builds on

19 of 28 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  9. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  10. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  11. 2018CoQA: A Conversational Question Answering Challengecited by 674
  12. 2021QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answeringcited by 386
7 more
  1. 2018QuAC : Question Answering in Contextcited by 379
  2. 2016Bidirectional Attention Flow for Machine Comprehensioncited by 278
  3. 2015Ask Me Anything: Dynamic Memory Networks for Natural Language Processingcited by 116
  4. 2016Dynamic Memory Networks for Visual and Textual Question Answeringcited by 62
  5. 2018QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationshipscited by 46
  6. 2019Careful Selection of Knowledge to solve Open Book Question Answeringcited by 35
  7. 2019Declarative Question Answering over Knowledge Bases containing Natural Language Text with Answer Set Programmingcited by 4

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 5.

6 papers.

  1. 2023Large Language Models for Biomedical Knowledge Graph Construction: Information extraction from EMR notescited by 17
  2. 2023Do Generative Large Language Models need billions of parameters?cited by 8
  3. 2023Does Synthetic Data Make Large Language Models More Efficient?cited by 8
  4. 2023Can pruning make Large Language Models more efficient?cited by 5
  5. 2022Alexa, Predict My Flight Delaycited by 4
  6. 2023Can a student Large Language Model perform as well as it's teacher?cited by 4