# RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records

arXiv: https://arxiv.org/abs/1805.10724 · HTML: https://mlsanity.com/paper/1805.10724/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2018.

## Built on by (most cited first, 30 of 61)

- [Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond](https://mlsanity.com/paper/2102.01998/) (2021)
- [Explaining Time Series Predictions with Dynamic Masks](https://mlsanity.com/paper/2106.05303/) (2021)
- [Deep learning for temporal data representation in electronic health records: A systematic review of challenges and methodologies](https://mlsanity.com/paper/2107.09951/) (2021)
- [The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations](https://mlsanity.com/paper/2212.11737/) (2022)
- [Visual Analytics For Machine Learning: A Data Perspective Survey](https://mlsanity.com/paper/2307.07712/) (2023)
- [Explainable AI for clinical and remote health applications: a survey on tabular and time series data](https://mlsanity.com/paper/2209.06528/) (2022)
- [A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?](https://mlsanity.com/paper/2304.04780/) (2023)
- [Sibyl: Understanding and Addressing the Usability Challenges of Machine Learning In High-Stakes Decision Making](https://mlsanity.com/paper/2103.02071/) (2021)
- [VisQA: X-raying Vision and Language Reasoning in Transformers](https://mlsanity.com/paper/2104.00926/) (2021)
- [A Unified Understanding of Deep NLP Models for Text Classification](https://mlsanity.com/paper/2206.09355/) (2022)
- [A Survey of Explainable Artificial Intelligence (XAI) in Financial Time Series Forecasting](https://mlsanity.com/paper/2407.15909/) (2024)
- [Multimodal data matters: language model pre-training over structured and unstructured electronic health records](https://mlsanity.com/paper/2201.10113/) (2022)
- [An explainable Transformer-based deep learning model for the prediction of incident heart failure](https://mlsanity.com/paper/2101.11359/) (2021)
- [VBridge: Connecting the Dots Between Features and Data to Explain Healthcare Models](https://mlsanity.com/paper/2108.02550/) (2021)
- [Targeted-BEHRT: Deep learning for observational causal inference on longitudinal electronic health records](https://mlsanity.com/paper/2202.03487/) (2022)
- [DeepSI: Interactive Deep Learning for Semantic Interaction](https://mlsanity.com/paper/2305.18357/) (2023)
- [Explainable Adversarial Attacks in Deep Neural Networks Using Activation Profiles](https://mlsanity.com/paper/2103.10229/) (2021)
- [Building Interpretable Models for Business Process Prediction using Shared and Specialised Attention Mechanisms](https://mlsanity.com/paper/2109.01419/) (2021)
- [Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks](https://mlsanity.com/paper/2502.10526/) (2025)
- [Applications of Explainable AI for 6G: Technical Aspects, Use Cases, and Research Challenges](https://mlsanity.com/paper/2112.04698/) (2021)
- [Explainable Artificial Intelligence Methods in Combating Pandemics: A Systematic Review](https://mlsanity.com/paper/2112.12705/) (2021)
- [Explainable AI applications in the Medical Domain: a systematic review](https://mlsanity.com/paper/2308.05411/) (2023)
- [ASAP: Interpretable Analysis and Summarization of AI-generated Image Patterns at Scale](https://mlsanity.com/paper/2404.02990/) (2024)
- [MedDiffusion: Boosting Health Risk Prediction via Diffusion-based Data Augmentation](https://mlsanity.com/paper/2310.02520/) (2023)
- [Modeling Long-term Dependencies and Short-term Correlations in Patient Journey Data with Temporal Attention Networks for Health Prediction](https://mlsanity.com/paper/2207.06414/) (2022)
- [Deep representation learning: Fundamentals, Perspectives, Applications, and Open Challenges](https://mlsanity.com/paper/2211.14732/) (2022)
- [TimeTuner: Diagnosing Time Representations for Time-Series Forecasting with Counterfactual Explanations](https://mlsanity.com/paper/2307.09916/) (2023)
- [Are We Closing the Loop Yet? Gaps in the Generalizability of VIS4ML Research](https://mlsanity.com/paper/2308.06290/) (2023)
- [Advancing Precision Oncology Through Modeling of Longitudinal and Multimodal Data](https://mlsanity.com/paper/2502.07836/) (2025)
- [THALIS: Human-Machine Analysis of Longitudinal Symptoms in Cancer Therapy](https://mlsanity.com/paper/2108.02817/) (2021)

## Abstract (arXiv)

We have recently seen many successful applications of recurrent neural networks (RNNs) on electronic medical records (EMRs), which contain histories of patients' diagnoses, medications, and other various events, in order to predict the current and future states of patients. Despite the strong performance of RNNs, it is often challenging for users to understand why the model makes a particular prediction. Such black-box nature of RNNs can impede its wide adoption in clinical practice. Furthermore, we have no established methods to interactively leverage users' domain expertise and prior knowledge as inputs for steering the model. Therefore, our design study aims to provide a visual analytics solution to increase interpretability and interactivity of RNNs via a joint effort of medical experts, artificial intelligence scientists, and visual analytics researchers. Following the iterative design process between the experts, we design, implement, and evaluate a visual analytics tool called RetainVis, which couples a newly improved, interpretable and interactive RNN-based model called RetainEX and visualizations for users' exploration of EMR data in the context of prediction tasks. Our study shows the effective use of RetainVis for gaining insights into how individual medical codes contribute to making risk predictions, using EMRs of patients with heart failure and cataract symptoms. Our study also demonstrates how we made substantial changes to the state-of-the-art RNN model called RETAIN in order to make use of temporal information and increase interactivity. This study will provide a useful guideline for researchers that aim to design an interpretable and interactive visual analytics tool for RNNs.