# SECNLP: A Survey of Embeddings in Clinical Natural Language Processing

arXiv: https://arxiv.org/abs/1903.01039 · HTML: https://mlsanity.com/paper/1903.01039/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2019.

## Built on by (most cited first, 16 of 16)

- [Pre-trained Language Models in Biomedical Domain: A Systematic Survey](https://mlsanity.com/paper/2110.05006/) (2021)
- [AMMU : A Survey of Transformer-based Biomedical Pretrained Language Models](https://mlsanity.com/paper/2105.00827/) (2021)
- [A Survey on Medical Document Summarization](https://mlsanity.com/paper/2212.01669/) (2022)
- [Open Challenges and Opportunities in Federated Foundation Models Towards Biomedical Healthcare](https://mlsanity.com/paper/2405.06784/) (2024)
- [On the Effectiveness of Compact Biomedical Transformers](https://mlsanity.com/paper/2209.03182/) (2022)
- [Generalizable and Scalable Multistage Biomedical Concept Normalization Leveraging Large Language Models](https://mlsanity.com/paper/2405.15122/) (2024)
- [Trustworthy AI: Securing Sensitive Data in Large Language Models](https://mlsanity.com/paper/2409.18222/) (2024)
- [Explainable Health Risk Predictor with Transformer-based Medicare Claim Encoder](https://mlsanity.com/paper/2105.09428/) (2021)
- [TransfoRhythm: A Transformer Architecture Conductive to Blood Pressure Estimation via Solo PPG Signal Capturing](https://mlsanity.com/paper/2404.15352/) (2024)
- [Integration of Domain Knowledge using Medical Knowledge Graph Deep Learning for Cancer Phenotyping](https://mlsanity.com/paper/2101.01337/) (2021)
- [Leveraging knowledge graphs to update scientific word embeddings using latent semantic imputation](https://mlsanity.com/paper/2210.15358/) (2022)
- [Toward a Neural Semantic Parsing System for EHR Question Answering](https://mlsanity.com/paper/2211.04569/) (2022)
- [Unlocking Electronic Health Records: A Hybrid Graph RAG Approach to Safe Clinical AI for Patient QA](https://mlsanity.com/paper/2602.00009/) (2025)
- [A reproducible experimental survey on biomedical sentence similarity: a string-based method sets the state of the art](https://mlsanity.com/paper/2205.08740/) (2022)
- [DeepProphet2 -- A Deep Learning Gene Recommendation Engine](https://mlsanity.com/paper/2208.01918/) (2022)
- [Language Models as Interfaces, Not Oracles: A Hybrid LLM-ML System for Pediatric Appendicitis](https://mlsanity.com/paper/2606.19183/) (2026)

## Abstract (arXiv)

Traditional representations like Bag of words are high dimensional, sparse and ignore the order as well as syntactic and semantic information. Distributed vector representations or embeddings map variable length text to dense fixed length vectors as well as capture the prior knowledge which can transferred to downstream tasks. Even though embedding has become de facto standard for representations in deep learning based NLP tasks in both general and clinical domains, there is no survey paper which presents a detailed review of embeddings in Clinical Natural Language Processing. In this survey paper, we discuss various medical corpora and their characteristics, medical codes and present a brief overview as well as comparison of popular embeddings models. We classify clinical embeddings into nine types and discuss each embedding type in detail. We discuss various evaluation methods followed by possible solutions to various challenges in clinical embeddings. Finally, we conclude with some of the future directions which will advance the research in clinical embeddings.