# Augmenting Biomedical Named Entity Recognition with General-domain Resources

arXiv: https://arxiv.org/abs/2406.10671 · HTML: https://mlsanity.com/paper/2406.10671/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 51 references that are in the map.
- First posted on arXiv 15 Jun 2024.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://mlsanity.com/paper/2210.10341/) (2022)
- [Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets](https://mlsanity.com/paper/1906.05474/) (2019)
- [SciFive: a text-to-text transformer model for biomedical literature](https://mlsanity.com/paper/2106.03598/) (2021)
- [BioRED: A Rich Biomedical Relation Extraction Dataset](https://mlsanity.com/paper/2204.04263/) (2022)
- [BioBART: Pretraining and Evaluation of A Biomedical Generative Language Model](https://mlsanity.com/paper/2204.03905/) (2022)
- [Learning how to Active Learn: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/1708.02383/) (2017)
- [BERN2: an advanced neural biomedical named entity recognition and normalization tool](https://mlsanity.com/paper/2201.02080/) (2022)
- [Building a PubMed knowledge graph](https://mlsanity.com/paper/2005.04308/) (2020)
- [Pre-trained Language Model for Biomedical Question Answering](https://mlsanity.com/paper/1909.08229/) (2019)
- [Cross-type Biomedical Named Entity Recognition with Deep Multi-Task Learning](https://mlsanity.com/paper/1801.09851/) (2018)
- [BERT Based Clinical Knowledge Extraction for Biomedical Knowledge Graph Construction and Analysis](https://mlsanity.com/paper/2304.10996/) (2023)
- [MT-BioNER: Multi-task Learning for Biomedical Named Entity Recognition using Deep Bidirectional Transformers](https://mlsanity.com/paper/2001.08904/) (2020)
- [AIONER: All-in-one scheme-based biomedical named entity recognition using deep learning](https://mlsanity.com/paper/2211.16944/) (2022)
- [Answering Questions on COVID-19 in Real-Time](https://mlsanity.com/paper/2006.15830/) (2020)
- [Model Transfer for Tagging Low-resource Languages using a Bilingual Dictionary](https://mlsanity.com/paper/1705.00424/) (2017)
- [CollaboNet: collaboration of deep neural networks for biomedical named entity recognition](https://mlsanity.com/paper/1809.07950/) (2018)
- [How Do Your Biomedical Named Entity Recognition Models Generalize to Novel Entities?](https://mlsanity.com/paper/2101.00160/) (2021)
- [Vapur: A Search Engine to Find Related Protein-Compound Pairs in COVID-19 Literature](https://mlsanity.com/paper/2009.02526/) (2020)
- [Improving Tagging Consistency and Entity Coverage for Chemical Identification in Full-text Articles](https://mlsanity.com/paper/2111.10584/) (2021)

## Built on by (most cited first, 2 of 2)

- [Toward Federated Large Language Models in Medicine: A Parameter-Efficient Framework for Privacy-Preserving, Multi-Institutional Adaptation](https://mlsanity.com/paper/2601.22124/) (2026)
- [Effective Multi-Task Learning for Biomedical Named Entity Recognition](https://mlsanity.com/paper/2507.18542/) (2025)

## Abstract (arXiv)

Training a neural network-based biomedical named entity recognition (BioNER) model usually requires extensive and costly human annotations. While several studies have employed multi-task learning with multiple BioNER datasets to reduce human effort, this approach does not consistently yield performance improvements and may introduce label ambiguity in different biomedical corpora. We aim to tackle those challenges through transfer learning from easily accessible resources with fewer concept overlaps with biomedical datasets. We proposed GERBERA, a simple-yet-effective method that utilized general-domain NER datasets for training. We performed multi-task learning to train a pre-trained biomedical language model with both the target BioNER dataset and the general-domain dataset. Subsequently, we fine-tuned the models specifically for the BioNER dataset. We systematically evaluated GERBERA on five datasets of eight entity types, collectively consisting of 81,410 instances. Despite using fewer biomedical resources, our models demonstrated superior performance compared to baseline models trained with additional BioNER datasets. Specifically, our models consistently outperformed the baseline models in six out of eight entity types, achieving an average improvement of 0.9% over the best baseline performance across eight entities. Our method was especially effective in amplifying performance on BioNER datasets characterized by limited data, with a 4.7% improvement in F1 scores on the JNLPBA-RNA dataset. This study introduces a new training method that leverages cost-effective general-domain NER datasets to augment BioNER models. This approach significantly improves BioNER model performance, making it a valuable asset for scenarios with scarce or costly biomedical datasets.