# Explanation of Machine Learning Models Using Shapley Additive Explanation and Application for Real Data in Hospital

arXiv: https://arxiv.org/abs/2112.11071 · HTML: https://mlsanity.com/paper/2112.11071/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 14 references that are in the map.
- First posted on arXiv 21 Dec 2021.

## Builds on (most cited first, 3 of 3)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Consistent Individualized Feature Attribution for Tree Ensembles](https://mlsanity.com/paper/1802.03888/) (2018)

## Built on by (most cited first, 17 of 17)

- [Causal Representation Learning from Multimodal Clinical Records under Non-Random Modality Missingness](https://mlsanity.com/paper/2509.17228/) (2025)
- [Comparing Baseline Shapley and Integrated Gradients for Local Explanation: Some Additional Insights](https://mlsanity.com/paper/2208.06096/) (2022)
- [A Learning Search Algorithm for the Restricted Longest Common Subsequence Problem](https://mlsanity.com/paper/2410.12031/) (2024)
- [CHiRPE: A Step Towards Real-World Clinical NLP with Clinician-Oriented Model Explanations](https://mlsanity.com/paper/2601.18102/) (2026)
- [Alternative Methods to SHAP Derived from Properties of Kernels: A Note on Theoretical Analysis](https://mlsanity.com/paper/2406.00371/) (2024)
- [Decoding the human brain tissue response to radiofrequency excitation using a biophysical-model-free deep MRI on a chip framework](https://mlsanity.com/paper/2408.08376/) (2024)
- [Explainable AI for Mental Health Emergency Returns: Integrating LLMs with Predictive Modeling](https://mlsanity.com/paper/2502.00025/) (2025)
- [Hyperparameter Optimisation with Practical Interpretability and Explanation Methods in Probabilistic Curriculum Learning](https://mlsanity.com/paper/2504.06683/) (2025)
- [In defence of post-hoc explanations in medical AI](https://mlsanity.com/paper/2504.20741/) (2025)
- [Context-Aware Visualization for Explainable AI Recommendations in Social Media: A Vision for User-Aligned Explanations](https://mlsanity.com/paper/2508.00674/) (2025)
- [More Isn't Always Better: Balancing Decision Accuracy and Conformity Pressures in Multi-AI Advice](https://mlsanity.com/paper/2603.22152/) (2026)
- [Explanation of Machine Learning Models of Colon Cancer Using SHAP Considering Interaction Effects](https://mlsanity.com/paper/2208.03112/) (2022)
- [The Safety Challenges of Deep Learning in Real-World Type 1 Diabetes Management](https://mlsanity.com/paper/2310.14743/) (2023)
- [On-Demand Growth of Semiconductor Heterostructures Guided by Physics-Informed Machine Learning](https://mlsanity.com/paper/2408.03508/) (2024)
- [Developing Explainable Machine Learning Model using Augmented Concept Activation Vector](https://mlsanity.com/paper/2412.19208/) (2024)
- [Streamlining Admission with LOR Insights: AI-Based Leadership Assessment in Online Master's Program](https://mlsanity.com/paper/2508.05513/) (2025)
- [Explainable Multimodal Deep Learning Integrating Imaging and Clinical Data for Oral Potentially Malignant Disorder Detection](https://mlsanity.com/paper/2609.04512/) (2026)

## Abstract (arXiv)

When using machine learning techniques in decision-making processes, the interpretability of the models is important. In the present paper, we adopted the Shapley additive explanation (SHAP), which is based on fair profit allocation among many stakeholders depending on their contribution, for interpreting a gradient-boosting decision tree model using hospital data. For better interpretability, we propose two novel techniques as follows: (1) a new metric of feature importance using SHAP and (2) a technique termed feature packing, which packs multiple similar features into one grouped feature to allow an easier understanding of the model without reconstruction of the model. We then compared the explanation results between the SHAP framework and existing methods. In addition, we showed how the A/G ratio works as an important prognostic factor for cerebral infarction using our hospital data and proposed techniques.