# GRAM: Graph-based Attention Model for Healthcare Representation Learning

arXiv: https://arxiv.org/abs/1611.07012 · HTML: https://mlsanity.com/paper/1611.07012/

## Key facts

- Cited by 156 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2016.

## Built on by (most cited first, 30 of 156)

- [EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Models](https://mlsanity.com/paper/2307.02028/) (2023)
- [Graph Representation Learning in Biomedicine](https://mlsanity.com/paper/2104.04883/) (2021)
- [Time Series Prediction using Deep Learning Methods in Healthcare](https://mlsanity.com/paper/2108.13461/) (2021)
- [GraphCare: Enhancing Healthcare Predictions with Personalized Knowledge Graphs](https://mlsanity.com/paper/2305.12788/) (2023)
- [HINT: Hierarchical Interaction Network for Trial Outcome Prediction Leveraging Web Data](https://mlsanity.com/paper/2102.04252/) (2021)
- [A Survey on Graph-Based Deep Learning for Computational Histopathology](https://mlsanity.com/paper/2107.00272/) (2021)
- [Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions](https://mlsanity.com/paper/2107.13782/) (2021)
- [FedGraph: Federated Graph Learning with Intelligent Sampling](https://mlsanity.com/paper/2111.01370/) (2021)
- [Heterogeneous Similarity Graph Neural Network on Electronic Health Records](https://mlsanity.com/paper/2101.06800/) (2021)
- [Collaborative Graph Learning with Auxiliary Text for Temporal Event Prediction in Healthcare](https://mlsanity.com/paper/2105.07542/) (2021)
- [Rethinking Human-AI Collaboration in Complex Medical Decision Making: A Case Study in Sepsis Diagnosis](https://mlsanity.com/paper/2309.12368/) (2023)
- [Unifying Heterogeneous Electronic Health Records Systems via Text-Based Code Embedding](https://mlsanity.com/paper/2108.03625/) (2021)
- [REALM: RAG-Driven Enhancement of Multimodal Electronic Health Records Analysis via Large Language Models](https://mlsanity.com/paper/2402.07016/) (2024)
- [ColaCare: Enhancing Electronic Health Record Modeling through Large Language Model-Driven Multi-Agent Collaboration](https://mlsanity.com/paper/2410.02551/) (2024)
- [RAM-EHR: Retrieval Augmentation Meets Clinical Predictions on Electronic Health Records](https://mlsanity.com/paper/2403.00815/) (2024)
- [Context-aware Health Event Prediction via Transition Functions on Dynamic Disease Graphs](https://mlsanity.com/paper/2112.05195/) (2021)
- [Multimodal Data Integration for Oncology in the Era of Deep Neural Networks: A Review](https://mlsanity.com/paper/2303.06471/) (2023)
- [DeepOnto: A Python Package for Ontology Engineering with Deep Learning](https://mlsanity.com/paper/2307.03067/) (2023)
- [Uncertainty Quantification on Clinical Trial Outcome Prediction](https://mlsanity.com/paper/2401.03482/) (2024)
- [MentorGNN: Deriving Curriculum for Pre-Training GNNs](https://mlsanity.com/paper/2208.09905/) (2022)
- [A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applications](https://mlsanity.com/paper/2404.14809/) (2024)
- [EMERGE: Enhancing Multimodal Electronic Health Records Predictive Modeling with Retrieval-Augmented Generation](https://mlsanity.com/paper/2406.00036/) (2024)
- [Multi-Label Clinical Time-Series Generation via Conditional GAN](https://mlsanity.com/paper/2204.04797/) (2022)
- [Explainable Machine Learning with Prior Knowledge: An Overview](https://mlsanity.com/paper/2105.10172/) (2021)
- [Self-Supervised Graph Learning with Hyperbolic Embedding for Temporal Health Event Prediction](https://mlsanity.com/paper/2106.04751/) (2021)
- [Deep Efficient Private Neighbor Generation for Subgraph Federated Learning](https://mlsanity.com/paper/2401.04336/) (2024)
- [Multimodal data matters: language model pre-training over structured and unstructured electronic health records](https://mlsanity.com/paper/2201.10113/) (2022)
- [SPOT: Sequential Predictive Modeling of Clinical Trial Outcome with Meta-Learning](https://mlsanity.com/paper/2304.05352/) (2023)
- [A Review on Knowledge Graphs for Healthcare: Resources, Applications, and Promises](https://mlsanity.com/paper/2306.04802/) (2023)
- [Sequential Diagnosis Prediction with Transformer and Ontological Representation](https://mlsanity.com/paper/2109.03069/) (2021)

## Abstract (arXiv)

Deep learning methods exhibit promising performance for predictive modeling in healthcare, but two important challenges remain: -Data insufficiency:Often in healthcare predictive modeling, the sample size is insufficient for deep learning methods to achieve satisfactory results. -Interpretation:The representations learned by deep learning methods should align with medical knowledge. To address these challenges, we propose a GRaph-based Attention Model, GRAM that supplements electronic health records (EHR) with hierarchical information inherent to medical ontologies. Based on the data volume and the ontology structure, GRAM represents a medical concept as a combination of its ancestors in the ontology via an attention mechanism. We compared predictive performance (i.e. accuracy, data needs, interpretability) of GRAM to various methods including the recurrent neural network (RNN) in two sequential diagnoses prediction tasks and one heart failure prediction task. Compared to the basic RNN, GRAM achieved 10% higher accuracy for predicting diseases rarely observed in the training data and 3% improved area under the ROC curve for predicting heart failure using an order of magnitude less training data. Additionally, unlike other methods, the medical concept representations learned by GRAM are well aligned with the medical ontology. Finally, GRAM exhibits intuitive attention behaviors by adaptively generalizing to higher level concepts when facing data insufficiency at the lower level concepts.