# Medical Concept Embedding with Time-Aware Attention

arXiv: https://arxiv.org/abs/1806.02873 · HTML: https://mlsanity.com/paper/1806.02873/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2018.

## Built on by (most cited first, 11 of 11)

- [Multi-Time Attention Networks for Irregularly Sampled Time Series](https://mlsanity.com/paper/2101.10318/) (2021)
- [Heterogeneous Similarity Graph Neural Network on Electronic Health Records](https://mlsanity.com/paper/2101.06800/) (2021)
- [Natural Language Processing for Smart Healthcare](https://mlsanity.com/paper/2110.15803/) (2021)
- [Sequential Diagnosis Prediction with Transformer and Ontological Representation](https://mlsanity.com/paper/2109.03069/) (2021)
- [DeepJoint: Robust Survival Modelling Under Clinical Presence Shift](https://mlsanity.com/paper/2205.13481/) (2022)
- [ICE-NODE: Integration of Clinical Embeddings with Neural Ordinary Differential Equations](https://mlsanity.com/paper/2207.01873/) (2022)
- [sEHR-CE: Language modelling of structured EHR data for efficient and generalizable patient cohort expansion](https://mlsanity.com/paper/2211.17121/) (2022)
- [Prediction of Survival Outcomes under Clinical Presence Shift: A Joint Neural Network Architecture](https://mlsanity.com/paper/2508.05472/) (2025)
- [Transfer Learning in Electronic Health Records through Clinical Concept Embedding](https://mlsanity.com/paper/2107.12919/) (2021)
- [Guardrails for avoiding harmful medical product recommendations and off-label promotion in generative AI models](https://mlsanity.com/paper/2406.16455/) (2024)
- [MATA-Former & SIICU: Semantic Aware Temporal Alignment for High-Fidelity ICU Risk Prediction](https://mlsanity.com/paper/2604.01727/) (2026)

## Abstract (arXiv)

Embeddings of medical concepts such as medication, procedure and diagnosis codes in Electronic Medical Records (EMRs) are central to healthcare analytics. Previous work on medical concept embedding takes medical concepts and EMRs as words and documents respectively. Nevertheless, such models miss out the temporal nature of EMR data. On the one hand, two consecutive medical concepts do not indicate they are temporally close, but the correlations between them can be revealed by the time gap. On the other hand, the temporal scopes of medical concepts often vary greatly (e.g., textitcommon cold and textitdiabetes). In this paper, we propose to incorporate the temporal information to embed medical codes. Based on the Continuous Bag-of-Words model, we employ the attention mechanism to learn a "soft" time-aware context window for each medical concept. Experiments on public and proprietary datasets through clustering and nearest neighbour search tasks demonstrate the effectiveness of our model, showing that it outperforms five state-of-the-art baselines.