mlsanityData to 7 Oct 2026

Pre-trained Language Model for Biomedical Question Answering

arXiv 1909.08229 ↗First posted 18 Sep 2019cs.CLcited by 31

Not on the map. Why?

Abstract (arXiv)

The recent success of question answering systems is largely attributed to pre-trained language models. However, as language models are mostly pre-trained on general domain corpora such as Wikipedia, they often have difficulty in understanding biomedical questions. In this paper, we investigate the performance of BioBERT, a pre-trained biomedical language model, in answering biomedical questions including factoid, list, and yes/no type questions. BioBERT uses almost the same structure across various question types and achieved the best performance in the 7th BioASQ Challenge (Task 7b, Phase B). BioBERT pre-trained on SQuAD or SQuAD 2.0 easily outperformed previous state-of-the-art models. BioBERT obtains the best performance when it uses the appropriate pre-/post-processing strategies for questions, passages, and answers.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 13; 2022: 5; 2023: 8; 2024: 5.

31 papers.

Sort:

By area:

  1. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  2. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 123
  3. 2022Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysiscited by 104
  4. 2021Pre-trained Language Models in Biomedical Domain: A Systematic Surveycited by 87
  5. 2021Biomedical Question Answering: A Survey of Approaches and Challengescited by 46
  6. 2021Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Reviewcited by 35
  7. 2021AMMU : A Survey of Transformer-based Biomedical Pretrained Language Modelscited by 32
  8. 2023Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answeringcited by 28
  9. 2022Paper Plain: Making Medical Research Papers Approachable to Healthcare Consumers with Natural Language Processingcited by 23
  10. 2021Boosting Low-Resource Biomedical QA via Entity-Aware Masking Strategiescited by 21
  11. 2023Transformers in Healthcare: A Surveycited by 17
  12. 2021Overview of BioASQ 2020: The eighth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answeringcited by 13
19 more
  1. 2021Sequence tagging for biomedical extractive question answeringcited by 12
  2. 2021ELECTRAMed: a new pre-trained language representation model for biomedical NLPcited by 11
  3. 2023The Semantic Reader Project: Augmenting Scholarly Documents through AI-Powered Interactive Reading Interfacescited by 9
  4. 2023Informing clinical assessment by contextualizing post-hoc explanations of risk prediction models in type-2 diabetescited by 8
  5. 2021Detect and Classify -- Joint Span Detection and Classification for Health Outcomescited by 6
  6. 2022Data Augmentation for Biomedical Factoid Question Answeringcited by 5
  7. 2023Impact of Tokenization on LLaMa Russian Adaptationcited by 5
  8. 2024Knowledge of Pretrained Language Models on Surface Information of Tokenscited by 5
  9. 2022BioADAPT-MRC: Adversarial Learning-based Domain Adaptation Improves Biomedical Machine Reading Comprehension Taskcited by 3
  10. 2024Pre-Trained Language Models for Keyphrase Prediction: A Reviewcited by 3
  11. 2024Augmenting Biomedical Named Entity Recognition with General-domain Resourcescited by 2
  12. 2022Science Checker: Extractive-Boolean Question Answering For Scientific Fact Checkingcited by 1
  13. 2021Let Your Heart Speak in its Mother Tongue: Multilingual Captioning of Cardiac Signalscited by 0
  14. 2021Transformer-based Language Models for Factoid Question Answering at BioASQ9bcited by 0
  15. 2021Recent Advances in Automated Question Answering In Biomedical Domaincited by 0
  16. 2023Using Weak Supervision and Data Augmentation in Question Answeringcited by 0
  17. 2023Fine-tuning pre-trained extractive QA models for clinical document parsingcited by 0
  18. 2024A Question Answering Based Pipeline for Comprehensive Chinese EHR Information Extractioncited by 0
  19. 2024Question answering system of bridge design specification based on large language modelcited by 0