# MT-BioNER: Multi-task Learning for Biomedical Named Entity Recognition using Deep Bidirectional Transformers

arXiv: https://arxiv.org/abs/2001.08904 · HTML: https://mlsanity.com/paper/2001.08904/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jan 2020.

## Built on by (most cited first, 15 of 15)

- [AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing](https://mlsanity.com/paper/2108.05542/) (2021)
- [Pre-trained Language Models in Biomedical Domain: A Systematic Survey](https://mlsanity.com/paper/2110.05006/) (2021)
- [AMMU : A Survey of Transformer-based Biomedical Pretrained Language Models](https://mlsanity.com/paper/2105.00827/) (2021)
- [Transformers in Healthcare: A Survey](https://mlsanity.com/paper/2307.00067/) (2023)
- [CLIN-X: pre-trained language models and a study on cross-task transfer for concept extraction in the clinical domain](https://mlsanity.com/paper/2112.08754/) (2021)
- [Comparison of biomedical relationship extraction methods and models for knowledge graph creation](https://mlsanity.com/paper/2201.01647/) (2022)
- [Multilingual Clinical NER: Translation or Cross-lingual Transfer?](https://mlsanity.com/paper/2306.04384/) (2023)
- [Large Language Model-based Role-Playing for Personalized Medical Jargon Extraction](https://mlsanity.com/paper/2408.05555/) (2024)
- [MTLSegFormer: Multi-task Learning with Transformers for Semantic Segmentation in Precision Agriculture](https://mlsanity.com/paper/2305.02813/) (2023)
- [Fine-Tuning BERT for Automatic ADME Semantic Labeling in FDA Drug Labeling to Enhance Product-Specific Guidance Assessment](https://mlsanity.com/paper/2207.12376/) (2022)
- [Augmenting Biomedical Named Entity Recognition with General-domain Resources](https://mlsanity.com/paper/2406.10671/) (2024)
- [Multilingual Name Entity Recognition and Intent Classification Employing Deep Learning Architectures](https://mlsanity.com/paper/2211.02415/) (2022)
- [Enhancing Biomedical Named Entity Recognition via Multiple Programming Languages Instruction Tuning and Ensemble Method](https://mlsanity.com/paper/2610.02949/) (2026)
- [rx-anon -- A Novel Approach on the De-Identification of Heterogeneous Data based on a Modified Mondrian Algorithm](https://mlsanity.com/paper/2105.08842/) (2021)
- [Effective Multi-Task Learning for Biomedical Named Entity Recognition](https://mlsanity.com/paper/2507.18542/) (2025)

## Abstract (arXiv)

Conversational agents such as Cortana, Alexa and Siri are continuously working on increasing their capabilities by adding new domains. The support of a new domain includes the design and development of a number of NLU components for domain classification, intents classification and slots tagging (including named entity recognition). Each component only performs well when trained on a large amount of labeled data. Second, these components are deployed on limited-memory devices which requires some model compression. Third, for some domains such as the health domain, it is hard to find a single training data set that covers all the required slot types. To overcome these mentioned problems, we present a multi-task transformer-based neural architecture for slot tagging. We consider the training of a slot tagger using multiple data sets covering different slot types as a multi-task learning problem. The experimental results on the biomedical domain have shown that the proposed approach outperforms the previous state-of-the-art systems for slot tagging on the different benchmark biomedical datasets in terms of (time and memory) efficiency and effectiveness. The output slot tagger can be used by the conversational agent to better identify entities in the input utterances.