# Concept-based model explanations for Electronic Health Records

arXiv: https://arxiv.org/abs/2012.02308 · HTML: https://mlsanity.com/paper/2012.02308/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2020.

## Built on by (most cited first, 14 of 14)

- [Acquisition of Chess Knowledge in AlphaZero](https://mlsanity.com/paper/2111.09259/) (2021)
- [Concept Activation Regions: A Generalized Framework For Concept-Based Explanations](https://mlsanity.com/paper/2209.11222/) (2022)
- [Human-Centered Concept Explanations for Neural Networks](https://mlsanity.com/paper/2202.12451/) (2022)
- [Explainable AI for Time Series via Virtual Inspection Layers](https://mlsanity.com/paper/2303.06365/) (2023)
- [Best of both worlds: local and global explanations with human-understandable concepts](https://mlsanity.com/paper/2106.08641/) (2021)
- [Making Corgis Important for Honeycomb Classification: Adversarial Attacks on Concept-based Explainability Tools](https://mlsanity.com/paper/2110.07120/) (2021)
- [Towards Benchmarking Explainable Artificial Intelligence Methods](https://mlsanity.com/paper/2208.12120/) (2022)
- [A Self-explaining Neural Architecture for Generalizable Concept Learning](https://mlsanity.com/paper/2405.00349/) (2024)
- [C-SHAP for time series: An approach to high-level temporal explanations](https://mlsanity.com/paper/2504.11159/) (2025)
- [Self-explaining Neural Network with Concept-based Explanations for ICU Mortality Prediction](https://mlsanity.com/paper/2110.04598/) (2021)
- [Explaining Deep Neural Networks for Bearing Fault Detection with Vibration Concepts](https://mlsanity.com/paper/2310.11450/) (2023)
- [Boosting the interpretability of clinical risk scores with intervention predictions](https://mlsanity.com/paper/2207.02941/) (2022)
- [Mapping Knowledge Representations to Concepts: A Review and New Perspectives](https://mlsanity.com/paper/2301.00189/) (2022)
- [A Unified Concept-Based System for Local, Global, and Misclassification Explanations](https://mlsanity.com/paper/2306.03531/) (2023)

## Abstract (arXiv)

Recurrent Neural Networks (RNNs) are often used for sequential modeling of adverse outcomes in electronic health records (EHRs) due to their ability to encode past clinical states. These deep, recurrent architectures have displayed increased performance compared to other modeling approaches in a number of tasks, fueling the interest in deploying deep models in clinical settings. One of the key elements in ensuring safe model deployment and building user trust is model explainability. Testing with Concept Activation Vectors (TCAV) has recently been introduced as a way of providing human-understandable explanations by comparing high-level concepts to the network's gradients. While the technique has shown promising results in real-world imaging applications, it has not been applied to structured temporal inputs. To enable an application of TCAV to sequential predictions in the EHR, we propose an extension of the method to time series data. We evaluate the proposed approach on an open EHR benchmark from the intensive care unit, as well as synthetic data where we are able to better isolate individual effects.