# Neural Domain Adaptation for Biomedical Question Answering

arXiv: https://arxiv.org/abs/1706.03610 · HTML: https://mlsanity.com/paper/1706.03610/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2017.

## Built on by (most cited first, 30 of 30)

- [MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound](https://mlsanity.com/paper/2201.02639/) (2022)
- [How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness?](https://mlsanity.com/paper/2112.11668/) (2021)
- [Biomedical Question Answering: A Survey of Approaches and Challenges](https://mlsanity.com/paper/2102.05281/) (2021)
- [Scaling Laws for Forgetting When Fine-Tuning Large Language Models](https://mlsanity.com/paper/2401.05605/) (2024)
- [Preserving Pre-trained Features Helps Calibrate Fine-tuned Language Models](https://mlsanity.com/paper/2305.19249/) (2023)
- [Refining Language Models with Compositional Explanations](https://mlsanity.com/paper/2103.10415/) (2021)
- [Self-Supervised Test-Time Learning for Reading Comprehension](https://mlsanity.com/paper/2103.11263/) (2021)
- [Learning to Transfer Prompts for Text Generation](https://mlsanity.com/paper/2205.01543/) (2022)
- [Why is constrained neural language generation particularly challenging?](https://mlsanity.com/paper/2206.05395/) (2022)
- [Sequence tagging for biomedical extractive question answering](https://mlsanity.com/paper/2104.07535/) (2021)
- [Synthetic Question Value Estimation for Domain Adaptation of Question Answering](https://mlsanity.com/paper/2203.08926/) (2022)
- [Studying Catastrophic Forgetting in Neural Ranking Models](https://mlsanity.com/paper/2101.06984/) (2021)
- [Bioformer: an efficient transformer language model for biomedical text mining](https://mlsanity.com/paper/2302.01588/) (2023)
- [On the Usability of Transformers-based models for a French Question-Answering task](https://mlsanity.com/paper/2207.09150/) (2022)
- [Backward Compatibility During Data Updates by Weight Interpolation](https://mlsanity.com/paper/2301.10546/) (2023)
- [Synthetic Target Domain Supervision for Open Retrieval QA](https://mlsanity.com/paper/2204.09248/) (2022)
- [Applications of Large Models in Medicine](https://mlsanity.com/paper/2502.17132/) (2025)
- [A Neural Question Answering System for Basic Questions about Subroutines](https://mlsanity.com/paper/2101.03999/) (2021)
- [Task-adaptive Pre-training of Language Models with Word Embedding Regularization](https://mlsanity.com/paper/2109.08354/) (2021)
- [Rumour Detection via Zero-shot Cross-lingual Transfer Learning](https://mlsanity.com/paper/2109.12773/) (2021)
- [Continual Learning of Long Topic Sequences in Neural Information Retrieval](https://mlsanity.com/paper/2201.03356/) (2022)
- [Fine-tuning Large Language Models with Limited Data: A Survey and Practical Guide](https://mlsanity.com/paper/2411.09539/) (2024)
- [Question answering systems for health professionals at the point of care -- a systematic review](https://mlsanity.com/paper/2402.01700/) (2024)
- [Neural Supervised Domain Adaptation by Augmenting Pre-trained Models with Random Units](https://mlsanity.com/paper/2106.04935/) (2021)
- [Task Transfer and Domain Adaptation for Zero-Shot Question Answering](https://mlsanity.com/paper/2206.06705/) (2022)
- [Supervised In-Context Fine-Tuning for Generative Sequence Labeling](https://mlsanity.com/paper/2509.00921/) (2025)
- [Transformer-based Language Models for Factoid Question Answering at BioASQ9b](https://mlsanity.com/paper/2109.07185/) (2021)
- [Recent Advances in Automated Question Answering In Biomedical Domain](https://mlsanity.com/paper/2111.05937/) (2021)
- [DPRK-BERT: The Supreme Language Model](https://mlsanity.com/paper/2112.00567/) (2021)
- [Domain-specific or Uncertainty-aware models: Does it really make a difference for biomedical text classification?](https://mlsanity.com/paper/2407.12626/) (2024)

## Abstract (arXiv)

Factoid question answering (QA) has recently benefited from the development of deep learning (DL) systems. Neural network models outperform traditional approaches in domains where large datasets exist, such as SQuAD (ca. 100,000 questions) for Wikipedia articles. However, these systems have not yet been applied to QA in more specific domains, such as biomedicine, because datasets are generally too small to train a DL system from scratch. For example, the BioASQ dataset for biomedical QA comprises less then 900 factoid (single answer) and list (multiple answers) QA instances. In this work, we adapt a neural QA system trained on a large open-domain dataset (SQuAD, source) to a biomedical dataset (BioASQ, target) by employing various transfer learning techniques. Our network architecture is based on a state-of-the-art QA system, extended with biomedical word embeddings and a novel mechanism to answer list questions. In contrast to existing biomedical QA systems, our system does not rely on domain-specific ontologies, parsers or entity taggers, which are expensive to create. Despite this fact, our systems achieve state-of-the-art results on factoid questions and competitive results on list questions.