mlsanityData to 7 Oct 2026

Transformer-Based Models for Question Answering on COVID19

arXiv 2101.11432 ↗First posted 16 Jan 2021cs.CL cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

In response to the Kaggle's COVID-19 Open Research Dataset (CORD-19) challenge, we have proposed three transformer-based question-answering systems using BERT, ALBERT, and T5 models. Since the CORD-19 dataset is unlabeled, we have evaluated the question-answering models' performance on two labeled questions answers datasets textemdash CovidQA and CovidGQA. The BERT-based QA system achieved the highest F1 score (26.32), while the ALBERT-based QA system achieved the highest Exact Match (13.04). However, numerous challenges are associated with developing high-performance question-answering systems for the ongoing COVID-19 pandemic and future pandemics. At the end of this paper, we discuss these challenges and suggest potential solutions to address them.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 15 of its 26 references that are in the map. First posted on arXiv 16 Jan 2021.

Builds on

15 of 26 references.

References · 15

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  9. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  10. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  11. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  12. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
3 more
  1. 2020Pretrained Transformers for Simple Question Answering over Knowledge Graphscited by 12
  2. 2019Advances in Natural Language Question Answering: A Reviewcited by 4
  3. 2020Non-Pharmaceutical Intervention Discovery with Topic Modelingcited by 2

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1; 2023: 4; 2024: 0; 2025: 1.

7 papers.

  1. 2021Pre-trained Language Models in Biomedical Domain: A Systematic Surveycited by 87
  2. 2023Explanatory Argument Extraction of Correct Answers in Resident Medical Examscited by 6
  3. 2023Core Building Blocks: Next Gen Geo Spatial GPT Applicationcited by 4
  4. 2022Doctor XAvIer: Explainable Diagnosis on Physician-Patient Dialogues and XAI Evaluationcited by 3
  5. 2023Numerical Reasoning for Financial Reportscited by 3
  6. 2023Audio-AdapterFusion: A Task-ID-free Approach for Efficient and Non-Destructive Multi-task Speech Recognitioncited by 0
  7. 2025Composable NLP Workflows for BERT-based Ranking and QA Systemcited by 0