# MIMIC-IF: Interpretability and Fairness Evaluation of Deep Learning Models on MIMIC-IV Dataset

arXiv: https://arxiv.org/abs/2102.06761 · HTML: https://mlsanity.com/paper/2102.06761/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 78 references that are in the map.
- First posted on arXiv 12 Feb 2021.

## Builds on (most cited first, 30 of 48)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias](https://mlsanity.com/paper/1810.01943/) (2018)
- [Multitask learning and benchmarking with clinical time series data](https://mlsanity.com/paper/1703.07771/) (2017)
- [Fairness without Demographics through Adversarially Reweighted Learning](https://mlsanity.com/paper/2006.13114/) (2020)
- [Recommendations as Treatments: Debiasing Learning and Evaluation](https://mlsanity.com/paper/1602.05352/) (2016)
- [Minimax Pareto Fairness: A Multi Objective Perspective](https://mlsanity.com/paper/2011.01821/) (2020)
- [Why Is My Classifier Discriminatory?](https://mlsanity.com/paper/1805.12002/) (2018)
- [The Shapley Taylor Interaction Index](https://mlsanity.com/paper/1902.05622/) (2019)
- [Fairness in Deep Learning: A Computational Perspective](https://mlsanity.com/paper/1908.08843/) (2019)
- [MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-III](https://mlsanity.com/paper/1907.08322/) (2019)
- [Invariant Representations without Adversarial Training](https://mlsanity.com/paper/1805.09458/) (2018)
- [Explaining Explanations: Axiomatic Feature Interactions for Deep Networks](https://mlsanity.com/paper/2002.04138/) (2020)

## Built on by (most cited first, 15 of 15)

- [A Consistent and Efficient Evaluation Strategy for Attribution Methods](https://mlsanity.com/paper/2202.00449/) (2022)
- [Integrated multimodal artificial intelligence framework for healthcare applications](https://mlsanity.com/paper/2202.12998/) (2022)
- [Synthetic Benchmarks for Scientific Research in Explainable Machine Learning](https://mlsanity.com/paper/2106.12543/) (2021)
- [AMMU : A Survey of Transformer-based Biomedical Pretrained Language Models](https://mlsanity.com/paper/2105.00827/) (2021)
- [Algorithm Fairness in AI for Medicine and Healthcare](https://mlsanity.com/paper/2110.00603/) (2021)
- [Pitfalls of Explainable ML: An Industry Perspective](https://mlsanity.com/paper/2106.07758/) (2021)
- [Sampling Through the Lens of Sequential Decision Making](https://mlsanity.com/paper/2208.08056/) (2022)
- [Uncovering Bias in Personal Informatics](https://mlsanity.com/paper/2303.15592/) (2023)
- [Consolidated learning -- a domain-specific model-free optimization strategy with examples for XGBoost and MIMIC-IV](https://mlsanity.com/paper/2201.11815/) (2022)
- [MLHOps: Machine Learning for Healthcare Operations](https://mlsanity.com/paper/2305.02474/) (2023)
- [Explainable AI for clinical risk prediction: a survey of concepts, methods, and modalities](https://mlsanity.com/paper/2308.08407/) (2023)
- [Unsupervised Learning to Subphenotype Delirium Patients from Electronic Health Records](https://mlsanity.com/paper/2111.00592/) (2021)
- [Fair Machine Learning in Healthcare: A Review](https://mlsanity.com/paper/2206.14397/) (2022)
- [CT-HEG: A Bidirectional, Timestamp-Attributed Event Graph for ICU In-Hospital Mortality Prediction - An Architectural Ablation Study](https://mlsanity.com/paper/2608.02663/) (2026)
- [Beyond Demographic Balance: Multi-Metric and Intersectional Evaluation of Fairness in MIMIC-IV Mortality Prediction](https://mlsanity.com/paper/2610.01645/) (2026)

## Abstract (arXiv)

The recent release of large-scale healthcare datasets has greatly propelled the research of data-driven deep learning models for healthcare applications. However, due to the nature of such deep black-boxed models, concerns about interpretability, fairness, and biases in healthcare scenarios where human lives are at stake call for a careful and thorough examinations of both datasets and models. In this work, we focus on MIMIC-IV (Medical Information Mart for Intensive Care, version IV), the largest publicly available healthcare dataset, and conduct comprehensive analyses of dataset representation bias as well as interpretability and prediction fairness of deep learning models for in-hospital mortality prediction. In terms of interpretabilty, we observe that (1) the best performing interpretability method successfully identifies critical features for mortality prediction on various prediction models; (2) demographic features are important for prediction. In terms of fairness, we observe that (1) there exists disparate treatment in prescribing mechanical ventilation among patient groups across ethnicity, gender and age; (2) all of the studied mortality predictors are generally fair while the IMV-LSTM (Interpretable Multi-Variable Long Short-Term Memory) model provides the most accurate and unbiased predictions across all protected groups. We further draw concrete connections between interpretability methods and fairness metrics by showing how feature importance from interpretability methods can be beneficial in quantifying potential disparities in mortality predictors.