# Clinical Concept Embeddings Learned from Massive Sources of Multimodal Medical Data

arXiv: https://arxiv.org/abs/1804.01486 · HTML: https://mlsanity.com/paper/1804.01486/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Apr 2018.

## Built on by (most cited first, 30 of 35)

- [Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review](https://mlsanity.com/paper/2403.02469/) (2024)
- [CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction tasks](https://mlsanity.com/paper/2111.08585/) (2021)
- [Generating Scientific Claims for Zero-Shot Scientific Fact Checking](https://mlsanity.com/paper/2203.12990/) (2022)
- [Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Review](https://mlsanity.com/paper/2107.02975/) (2021)
- [EHRMamba: Towards Generalizable and Scalable Foundation Models for Electronic Health Records](https://mlsanity.com/paper/2405.14567/) (2024)
- [Multimodal Learning on Graphs for Disease Relation Extraction](https://mlsanity.com/paper/2203.08893/) (2022)
- [Natural Language Processing for Smart Healthcare](https://mlsanity.com/paper/2110.15803/) (2021)
- [Multimodal data matters: language model pre-training over structured and unstructured electronic health records](https://mlsanity.com/paper/2201.10113/) (2022)
- [Multi-source Learning via Completion of Block-wise Overlapping Noisy Matrices](https://mlsanity.com/paper/2105.10360/) (2021)
- [Probing Pre-Trained Language Models for Disease Knowledge](https://mlsanity.com/paper/2106.07285/) (2021)
- [Bioformer: an efficient transformer language model for biomedical text mining](https://mlsanity.com/paper/2302.01588/) (2023)
- [A Hybrid Approach to Measure Semantic Relatedness in Biomedical Concepts](https://mlsanity.com/paper/2101.10196/) (2021)
- [Inference of Dependency Knowledge Graph for Electronic Health Records](https://mlsanity.com/paper/2312.15611/) (2023)
- [Representation learning to advance multi-institutional studies with electronic health record data from US and France](https://mlsanity.com/paper/2502.08547/) (2025)
- [Consensus Knowledge Graph Learning via Multi-view Sparse Low Rank Block Model](https://mlsanity.com/paper/2209.13762/) (2022)
- [Contrastive Learning on Multimodal Analysis of Electronic Health Records](https://mlsanity.com/paper/2403.14926/) (2024)
- [Integration of Large Language Models and Traditional Deep Learning for Social Determinants of Health Prediction](https://mlsanity.com/paper/2505.04655/) (2025)
- [Multi-modal Graph Learning over UMLS Knowledge Graphs](https://mlsanity.com/paper/2307.04461/) (2023)
- [TextEssence: A Tool for Interactive Analysis of Semantic Shifts Between Corpora](https://mlsanity.com/paper/2103.11029/) (2021)
- [Transfer Learning in Electronic Health Records through Clinical Concept Embedding](https://mlsanity.com/paper/2107.12919/) (2021)
- [Knowledge Graph Embeddings for Multi-Lingual Structured Representations of Radiology Reports](https://mlsanity.com/paper/2309.00917/) (2023)
- [Estimating Text Similarity based on Semantic Concept Embeddings](https://mlsanity.com/paper/2401.04422/) (2024)
- [Less Context, Same Performance: A RAG Framework for Resource-Efficient LLM-Based Clinical NLP](https://mlsanity.com/paper/2505.20320/) (2025)
- [KEEP: Integrating Medical Ontologies with Clinical Data for Robust Code Embeddings](https://mlsanity.com/paper/2510.05049/) (2025)
- [Predefined domain specific embeddings of food concepts and recipes: A case study on heterogeneous recipe datasets](https://mlsanity.com/paper/2302.01005/) (2023)
- [LATTE: Label-efficient Incident Phenotyping from Longitudinal Electronic Health Records](https://mlsanity.com/paper/2305.11407/) (2023)
- [Hierarchical Pretraining for Biomedical Term Embeddings](https://mlsanity.com/paper/2307.00266/) (2023)
- [Latent Factor Point Processes for Patient Representation in Electronic Health Records](https://mlsanity.com/paper/2508.20327/) (2025)
- [Automated Hierarchical Graph Construction for Multi-source Electronic Health Records](https://mlsanity.com/paper/2509.06576/) (2025)
- [A Common Pipeline for Harmonizing Electronic Health Record Data for Translational Research](https://mlsanity.com/paper/2509.08553/) (2025)

## Abstract (arXiv)

Word embeddings are a popular approach to unsupervised learning of word relationships that are widely used in natural language processing. In this article, we present a new set of embeddings for medical concepts learned using an extremely large collection of multimodal medical data. Leaning on recent theoretical insights, we demonstrate how an insurance claims database of 60 million members, a collection of 20 million clinical notes, and 1.7 million full text biomedical journal articles can be combined to embed concepts into a common space, resulting in the largest ever set of embeddings for 108,477 medical concepts. To evaluate our approach, we present a new benchmark methodology based on statistical power specifically designed to test embeddings of medical concepts. Our approach, called cui2vec, attains state-of-the-art performance relative to previous methods in most instances. Finally, we provide a downloadable set of pre-trained embeddings for other researchers to use, as well as an online tool for interactive exploration of the cui2vec embeddings