# Clinical Concept Extraction with Contextual Word Embedding

arXiv: https://arxiv.org/abs/1810.10566 · HTML: https://mlsanity.com/paper/1810.10566/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2018.

## Built on by (most cited first, 12 of 12)

- [SciFive: a text-to-text transformer model for biomedical literature](https://mlsanity.com/paper/2106.03598/) (2021)
- [Pre-trained Language Models in Biomedical Domain: A Systematic Survey](https://mlsanity.com/paper/2110.05006/) (2021)
- [LLMs Accelerate Annotation for Medical Information Extraction](https://mlsanity.com/paper/2312.02296/) (2023)
- [Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Review](https://mlsanity.com/paper/2107.02975/) (2021)
- [On the Calibration of Pre-trained Language Models using Mixup Guided by Area Under the Margin and Saliency](https://mlsanity.com/paper/2203.07559/) (2022)
- [Data Augmentations for Improved (Large) Language Model Generalization](https://mlsanity.com/paper/2310.12803/) (2023)
- [Hardness-guided domain adaptation to recognise biomedical named entities under low-resource scenarios](https://mlsanity.com/paper/2211.05980/) (2022)
- [A Hybrid Approach to Measure Semantic Relatedness in Biomedical Concepts](https://mlsanity.com/paper/2101.10196/) (2021)
- [A Novel Corpus of Annotated Medical Imaging Reports and Information Extraction Results Using BERT-based Language Models](https://mlsanity.com/paper/2403.18975/) (2024)
- [Disentangling Linguistic Features with Dimension-Wise Analysis of Vector Embeddings](https://mlsanity.com/paper/2504.14766/) (2025)
- [Jointly Learning Clinical Entities and Relations with Contextual Language Models and Explicit Context](https://mlsanity.com/paper/2102.11031/) (2021)
- [Comparative Analysis of Contextual Relation Extraction based on Deep Learning Models](https://mlsanity.com/paper/2309.06814/) (2023)

## Abstract (arXiv)

Automatic extraction of clinical concepts is an essential step for turning the unstructured data within a clinical note into structured and actionable information. In this work, we propose a clinical concept extraction model for automatic annotation of clinical problems, treatments, and tests in clinical notes utilizing domain-specific contextual word embedding. A contextual word embedding model is first trained on a corpus with a mixture of clinical reports and relevant Wikipedia pages in the clinical domain. Next, a bidirectional LSTM-CRF model is trained for clinical concept extraction using the contextual word embedding model. We tested our proposed model on the I2B2 2010 challenge dataset. Our proposed model achieved the best performance among reported baseline models and outperformed the state-of-the-art models by 3.4% in terms of F1-score.