# Interpretable and Explainable Machine Learning Methods for Predictive Process Monitoring: A Systematic Literature Review

arXiv: https://arxiv.org/abs/2312.17584 · HTML: https://mlsanity.com/paper/2312.17584/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 169 references that are in the map.
- First posted on arXiv 29 Dec 2023.

## Builds on (most cited first, 13 of 13)

- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Explaining Recurrent Neural Network Predictions in Sentiment Analysis](https://mlsanity.com/paper/1706.07206/) (2017)
- [Interpretable & Explorable Approximations of Black Box Models](https://mlsanity.com/paper/1707.01154/) (2017)
- [Explainable Predictive Process Monitoring](https://mlsanity.com/paper/2008.01807/) (2020)
- [Explainable Artificial Intelligence for Process Mining: A General Overview and Application of a Novel Local Explanation Approach for Predictive Process Monitoring](https://mlsanity.com/paper/2009.02098/) (2020)
- [Explainability Is in the Mind of the Beholder: Establishing the Foundations of Explainable Artificial Intelligence](https://mlsanity.com/paper/2112.14466/) (2021)
- [XNAP: Making LSTM-based Next Activity Predictions Explainable by Using LRP](https://mlsanity.com/paper/2008.07993/) (2020)
- [Quantifying and Explaining Machine Learning Uncertainty in Predictive Process Monitoring: An Operations Research Perspective](https://mlsanity.com/paper/2304.06412/) (2023)
- [An Explainable Decision Support System for Predictive Process Analytics](https://mlsanity.com/paper/2207.12782/) (2022)
- [Building Interpretable Models for Business Process Prediction using Shared and Specialised Attention Mechanisms](https://mlsanity.com/paper/2109.01419/) (2021)
- [Explainability in Process Outcome Prediction: Guidelines to Obtain Interpretable and Faithful Models](https://mlsanity.com/paper/2203.16073/) (2022)
- [Explainable AI Enabled Inspection of Business Process Prediction Models](https://mlsanity.com/paper/2107.09767/) (2021)
- [Communicating Uncertainty in Machine Learning Explanations: A Visualization Analytics Approach for Predictive Process Monitoring](https://mlsanity.com/paper/2304.05736/) (2023)

## Built on by (most cited first, 4 of 4)

- [XABPs: Towards eXplainable Autonomous Business Processes](https://mlsanity.com/paper/2507.23269/) (2025)
- [KANFIS: A Neuro-Symbolic Framework for Interpretable and Uncertainty-Aware Learning](https://mlsanity.com/paper/2602.03034/) (2026)
- [Interpretable and Explainable Surrogate Modeling for Simulations: A State-of-the-Art Survey and Perspectives on Explainable AI for Decision-Making](https://mlsanity.com/paper/2604.14240/) (2026)
- [GNN4PPM: Multi-Target Predictive Process Monitoring with Relational Graph Convolutional Networks](https://mlsanity.com/paper/2609.14534/) (2026)

## Abstract (arXiv)

This paper presents a systematic literature review (SLR) on the explainability and interpretability of machine learning (ML) models within the context of predictive process mining, using the PRISMA framework. Given the rapid advancement of artificial intelligence (AI) and ML systems, understanding the "black-box" nature of these technologies has become increasingly critical. Focusing specifically on the domain of process mining, this paper delves into the challenges of interpreting ML models trained with complex business process data. We differentiate between intrinsically interpretable models and those that require post-hoc explanation techniques, providing a comprehensive overview of the current methodologies and their applications across various application domains. Through a rigorous bibliographic analysis, this research offers a detailed synthesis of the state of explainability and interpretability in predictive process mining, identifying key trends, challenges, and future directions. Our findings aim to equip researchers and practitioners with a deeper understanding of how to develop and implement more trustworthy, transparent, and effective intelligent systems for predictive process analytics.