# RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism

arXiv: https://arxiv.org/abs/1608.05745 · HTML: https://mlsanity.com/paper/1608.05745/

## Key facts

- Cited by 349 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Aug 2016.

## Built on by (most cited first, 30 of 349)

- [Trustworthy AI: From Principles to Practices](https://mlsanity.com/paper/2110.01167/) (2021)
- [AdaRNN: Adaptive Learning and Forecasting of Time Series](https://mlsanity.com/paper/2108.04443/) (2021)
- [Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Series](https://mlsanity.com/paper/2107.14293/) (2021)
- [MOTOR: A Time-To-Event Foundation Model For Structured Medical Records](https://mlsanity.com/paper/2301.03150/) (2023)
- [Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond](https://mlsanity.com/paper/2102.01998/) (2021)
- [Time Series Prediction using Deep Learning Methods in Healthcare](https://mlsanity.com/paper/2108.13461/) (2021)
- [Explaining Time Series Predictions with Dynamic Masks](https://mlsanity.com/paper/2106.05303/) (2021)
- [GraphCare: Enhancing Healthcare Predictions with Personalized Knowledge Graphs](https://mlsanity.com/paper/2305.12788/) (2023)
- [Concept Activation Regions: A Generalized Framework For Concept-Based Explanations](https://mlsanity.com/paper/2209.11222/) (2022)
- [SafeDrug: Dual Molecular Graph Encoders for Recommending Effective and Safe Drug Combinations](https://mlsanity.com/paper/2105.02711/) (2021)
- [Hi-BEHRT: Hierarchical Transformer-based model for accurate prediction of clinical events using multimodal longitudinal electronic health records](https://mlsanity.com/paper/2106.11360/) (2021)
- [Integrating Expert ODEs into Neural ODEs: Pharmacology and Disease Progression](https://mlsanity.com/paper/2106.02875/) (2021)
- [Change Matters: Medication Change Prediction with Recurrent Residual Networks](https://mlsanity.com/paper/2105.01876/) (2021)
- [Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Review](https://mlsanity.com/paper/2107.02975/) (2021)
- [Deep learning for temporal data representation in electronic health records: A systematic review of challenges and methodologies](https://mlsanity.com/paper/2107.09951/) (2021)
- [CPLLM: Clinical Prediction with Large Language Models](https://mlsanity.com/paper/2309.11295/) (2023)
- [TimeX++: Learning Time-Series Explanations with Information Bottleneck](https://mlsanity.com/paper/2405.09308/) (2024)
- [On the Challenges and Opportunities in Generative AI](https://mlsanity.com/paper/2403.00025/) (2024)
- [Heterogeneous Similarity Graph Neural Network on Electronic Health Records](https://mlsanity.com/paper/2101.06800/) (2021)
- [Dynamic Gaussian Mixture based Deep Generative Model For Robust Forecasting on Sparse Multivariate Time Series](https://mlsanity.com/paper/2103.02164/) (2021)
- [Collaborative Graph Learning with Auxiliary Text for Temporal Event Prediction in Healthcare](https://mlsanity.com/paper/2105.07542/) (2021)
- [Conditional Generation Net for Medication Recommendation](https://mlsanity.com/paper/2202.06588/) (2022)
- [Rethinking Human-AI Collaboration in Complex Medical Decision Making: A Case Study in Sepsis Diagnosis](https://mlsanity.com/paper/2309.12368/) (2023)
- [Causal Interpretation of Self-Attention in Pre-Trained Transformers](https://mlsanity.com/paper/2310.20307/) (2023)
- [Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classification](https://mlsanity.com/paper/2105.02657/) (2021)
- [Temporal Dependencies in Feature Importance for Time Series Predictions](https://mlsanity.com/paper/2107.14317/) (2021)
- [Large Language Model Distilling Medication Recommendation Model](https://mlsanity.com/paper/2402.02803/) (2024)
- [SEAT: Stable and Explainable Attention](https://mlsanity.com/paper/2211.13290/) (2022)
- [ARM-Net: Adaptive Relation Modeling Network for Structured Data](https://mlsanity.com/paper/2107.01830/) (2021)
- [Unifying Heterogeneous Electronic Health Records Systems via Text-Based Code Embedding](https://mlsanity.com/paper/2108.03625/) (2021)

## Abstract (arXiv)

Accuracy and interpretability are two dominant features of successful predictive models. Typically, a choice must be made in favor of complex black box models such as recurrent neural networks (RNN) for accuracy versus less accurate but more interpretable traditional models such as logistic regression. This tradeoff poses challenges in medicine where both accuracy and interpretability are important. We addressed this challenge by developing the REverse Time AttentIoN model (RETAIN) for application to Electronic Health Records (EHR) data. RETAIN achieves high accuracy while remaining clinically interpretable and is based on a two-level neural attention model that detects influential past visits and significant clinical variables within those visits (e.g. key diagnoses). RETAIN mimics physician practice by attending the EHR data in a reverse time order so that recent clinical visits are likely to receive higher attention. RETAIN was tested on a large health system EHR dataset with 14 million visits completed by 263K patients over an 8 year period and demonstrated predictive accuracy and computational scalability comparable to state-of-the-art methods such as RNN, and ease of interpretability comparable to traditional models.