# Healthcare NER Models Using Language Model Pretraining

arXiv: https://arxiv.org/abs/1910.11241 · HTML: https://mlsanity.com/paper/1910.11241/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2019.

## Built on by (most cited first, 8 of 8)

- [Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Models](https://mlsanity.com/paper/2403.06448/) (2024)
- [Large Language Models for Medicine: A Survey](https://mlsanity.com/paper/2405.13055/) (2024)
- [Guiding Clinical Reasoning with Large Language Models via Knowledge Seeds](https://mlsanity.com/paper/2403.06609/) (2024)
- [Mitigating Entity-Level Hallucination in Large Language Models](https://mlsanity.com/paper/2407.09417/) (2024)
- [Review of Natural Language Processing in Pharmacology](https://mlsanity.com/paper/2208.10228/) (2022)
- [Out-of-Distribution Generalization in Text Classification: Past, Present, and Future](https://mlsanity.com/paper/2305.14104/) (2023)
- [Korean Bio-Medical Corpus (KBMC) for Medical Named Entity Recognition](https://mlsanity.com/paper/2403.16158/) (2024)
- [Utilizing AI and Social Media Analytics to Discover Adverse Side Effects of GLP-1 Receptor Agonists](https://mlsanity.com/paper/2404.01358/) (2024)

## Abstract (arXiv)

In this paper, we present our approach to extracting structured information from unstructured Electronic Health Records (EHR) [2] which can be used to, for example, study adverse drug reactions in patients due to chemicals in their products. Our solution uses a combination of Natural Language Processing (NLP) techniques and a web-based annotation tool to optimize the performance of a custom Named Entity Recognition (NER) [1] model trained on a limited amount of EHR training data. This work was presented at the first Health Search and Data Mining Workshop (HSDM 2020) [26]. We showcase a combination of tools and techniques leveraging the recent advancements in NLP aimed at targeting domain shifts by applying transfer learning and language model pre-training techniques [3]. We present a comparison of our technique to the current popular approaches and show the effective increase in performance of the NER model and the reduction in time to annotate data.A key observation of the results presented is that the F1 score of model (0.734) trained with our approach with just 50% of available training data outperforms the F1 score of the blank spaCy model without language model component (0.704) trained with 100% of the available training data. We also demonstrate an annotation tool to minimize domain expert time and the manual effort required to generate such a training dataset. Further, we plan to release the annotated dataset as well as the pre-trained model to the community to further research in medical health records.