mlsanityData to 14 Oct 2026

When to Fold'em: How to answer Unanswerable questions

arXiv 2105.00328 ↗First posted 1 May 2021cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

We present 3 different question-answering models trained on the SQuAD2.0 dataset -- BIDAF, DocumentQA and ALBERT Retro-Reader -- demonstrating the improvement of language models in the past three years. Through our research in fine-tuning pre-trained models for question-answering, we developed a novel approach capable of achieving a 2% point improvement in SQuAD2.0 F1 in reduced training time. Our method of re-initializing select layers of a parameter-shared language model is simple yet empirically powerful.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 18 of its 23 references that are in the map. First posted on arXiv 1 May 2021.

Builds on

18 of 23 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  9. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  10. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  11. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,127
  12. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 834
6 more
  1. 2016NewsQA: A Machine Comprehension Datasetcited by 331
  2. 2016Bidirectional Attention Flow for Machine Comprehensioncited by 278
  3. 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 225
  4. 2017Simple and Effective Multi-Paragraph Reading Comprehensioncited by 120
  5. 2020Retrospective Reader for Machine Reading Comprehensioncited by 86
  6. 2019A BERT Baseline for the Natural Questionscited by 33

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023To be or not to be? an exploration of continuously controllable prompt engineeringcited by 3