# Learning Conceptual-Contextual Embeddings for Medical Text

arXiv: https://arxiv.org/abs/1908.06203 · HTML: https://mlsanity.com/paper/1908.06203/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Aug 2019.

## Built on by (most cited first, 4 of 4)

- [Crafting Interpretable Embeddings by Asking LLMs Questions](https://mlsanity.com/paper/2405.16714/) (2024)
- [Textual Data Augmentation for Patient Outcomes Prediction](https://mlsanity.com/paper/2211.06778/) (2022)
- [Predicting Patient Readmission Risk from Medical Text via Knowledge Graph Enhanced Multiview Graph Convolution](https://mlsanity.com/paper/2201.02510/) (2021)
- [Enriching Unsupervised User Embedding via Medical Concepts](https://mlsanity.com/paper/2203.10627/) (2022)

## Abstract (arXiv)

External knowledge is often useful for natural language understanding tasks. We introduce a contextual text representation model called Conceptual-Contextual (CC) embeddings, which incorporates structured knowledge into text representations. Unlike entity embedding methods, our approach encodes a knowledge graph into a context model. CC embeddings can be easily reused for a wide range of tasks just like pre-trained language models. Our model effectively encodes the huge UMLS database by leveraging semantic generalizability. Experiments on electronic health records (EHRs) and medical text processing benchmarks showed our model gives a major boost to the performance of supervised medical NLP tasks.