# BioFLAIR: Pretrained Pooled Contextualized Embeddings for Biomedical Sequence Labeling Tasks

arXiv: https://arxiv.org/abs/1908.05760 · HTML: https://mlsanity.com/paper/1908.05760/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2019.

## Built on by (most cited first, 11 of 11)

- [A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics](https://mlsanity.com/paper/2310.05694/) (2023)
- [Pre-trained Language Models in Biomedical Domain: A Systematic Survey](https://mlsanity.com/paper/2110.05006/) (2021)
- [Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning](https://mlsanity.com/paper/2105.03654/) (2021)
- [ELECTRAMed: a new pre-trained language representation model for biomedical NLP](https://mlsanity.com/paper/2104.09585/) (2021)
- [ToNER: Type-oriented Named Entity Recognition with Generative Language Model](https://mlsanity.com/paper/2404.09145/) (2024)
- [A Survey of Pre-trained Language Models for Processing Scientific Text](https://mlsanity.com/paper/2401.17824/) (2024)
- [Regularization for Long Named Entity Recognition](https://mlsanity.com/paper/2104.07249/) (2021)
- [Assessment of contextualised representations in detecting outcome phrases in clinical trials](https://mlsanity.com/paper/2203.03547/) (2022)
- [NER-to-MRC: Named-Entity Recognition Completely Solving as Machine Reading Comprehension](https://mlsanity.com/paper/2305.03970/) (2023)
- [A ModelOps-based Framework for Intelligent Medical Knowledge Extraction](https://mlsanity.com/paper/2310.02593/) (2023)
- [Health Misinformation in Social Networks: A Survey of IT Approaches](https://mlsanity.com/paper/2410.18670/) (2024)

## Abstract (arXiv)

Biomedical Named Entity Recognition (NER) is a challenging problem in biomedical information processing due to the widespread ambiguity of out of context terms and extensive lexical variations. Performance on bioNER benchmarks continues to improve due to advances like BERT, GPT, and XLNet. FLAIR (1) is an alternative embedding model which is less computationally intensive than the others mentioned. We test FLAIR and its pretrained PubMed embeddings (which we term BioFLAIR) on a variety of bio NER tasks and compare those with results from BERT-type networks. We also investigate the effects of a small amount of additional pretraining on PubMed content, and of combining FLAIR and ELMO models. We find that with the provided embeddings, FLAIR performs on-par with the BERT networks - even establishing a new state of the art on one benchmark. Additional pretraining did not provide a clear benefit, although this might change with even more pretraining being done. Stacking the FLAIR embeddings with others typically does provide a boost in the benchmark results.