# Model Interpretation and Explainability: Towards Creating Transparency in Prediction Models

arXiv: https://arxiv.org/abs/2405.20794 · HTML: https://mlsanity.com/paper/2405.20794/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 23 references that are in the map.
- First posted on arXiv 31 May 2024.

## Builds on (most cited first, 10 of 10)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Model-Agnostic Interpretability of Machine Learning](https://mlsanity.com/paper/1606.05386/) (2016)
- [Towards better understanding of gradient-based attribution methods for Deep Neural Networks](https://mlsanity.com/paper/1711.06104/) (2017)
- [Global Explanations of Neural Networks: Mapping the Landscape of Predictions](https://mlsanity.com/paper/1902.02384/) (2019)
- [Towards Explainable Deep Learning for Credit Lending: A Case Study](https://mlsanity.com/paper/1811.06471/) (2018)

## Built on by (most cited first, 1 of 1)

- [RECAP: Transparent Inference-Time Emotion Alignment for Medical Dialogue Systems](https://mlsanity.com/paper/2509.10746/) (2025)

## Abstract (arXiv)

Explainable AI (XAI) has a counterpart in analytical modeling which we refer to as model explainability. We tackle the issue of model explainability in the context of prediction models. We analyze a dataset of loans from a credit card company and apply three stages: execute and compare four different prediction methods, apply the best known explainability techniques in the current literature to the model training sets to identify feature importance (FI) (static case), and finally to cross-check whether the FI set holds up under what if prediction scenarios for continuous and categorical variables (dynamic case). We found inconsistency in FI identification between the static and dynamic cases. We summarize the state of the art in model explainability and suggest further research to advance the field.