# Building Interpretable Models for Business Process Prediction using Shared and Specialised Attention Mechanisms

arXiv: https://arxiv.org/abs/2109.01419 · HTML: https://mlsanity.com/paper/2109.01419/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 46 references that are in the map.
- First posted on arXiv 3 Sep 2021.

## Builds on (most cited first, 17 of 17)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Is Attention Interpretable?](https://mlsanity.com/paper/1906.03731/) (2019)
- [RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism](https://mlsanity.com/paper/1608.05745/) (2016)
- [Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models](https://mlsanity.com/paper/1708.08296/) (2017)
- [Predictive Business Process Monitoring with LSTM Neural Networks](https://mlsanity.com/paper/1612.02130/) (2016)
- [RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records](https://mlsanity.com/paper/1805.10724/) (2018)
- [Predicting Process Behaviour using Deep Learning](https://mlsanity.com/paper/1612.04600/) (2016)
- [Deep Learning for Predictive Business Process Monitoring: Review and Benchmark](https://mlsanity.com/paper/2009.13251/) (2020)
- [Survey and cross-benchmark comparison of remaining time prediction methods in business process monitoring](https://mlsanity.com/paper/1805.02896/) (2018)
- [Outcome-Oriented Predictive Process Monitoring: Review and Benchmark](https://mlsanity.com/paper/1707.06766/) (2017)
- [ProcessTransformer: Predictive Business Process Monitoring with Transformer Network](https://mlsanity.com/paper/2104.00721/) (2021)
- [Explainable Predictive Process Monitoring](https://mlsanity.com/paper/2008.01807/) (2020)
- [Explainable Artificial Intelligence for Process Mining: A General Overview and Application of a Novel Local Explanation Approach for Predictive Process Monitoring](https://mlsanity.com/paper/2009.02098/) (2020)
- [XNAP: Making LSTM-based Next Activity Predictions Explainable by Using LRP](https://mlsanity.com/paper/2008.07993/) (2020)
- [Exploiting Event Log Event Attributes in RNN Based Prediction](https://mlsanity.com/paper/1904.06895/) (2019)

## Built on by (most cited first, 6 of 6)

- [An Explainable Decision Support System for Predictive Process Analytics](https://mlsanity.com/paper/2207.12782/) (2022)
- [Machine learning in business process management: A systematic literature review](https://mlsanity.com/paper/2405.16396/) (2024)
- [Interpretable and Explainable Machine Learning Methods for Predictive Process Monitoring: A Systematic Literature Review](https://mlsanity.com/paper/2312.17584/) (2023)
- [Analysis of business process automation as linear time-invariant system network](https://mlsanity.com/paper/2302.04785/) (2023)
- [The Role of Explanation Styles and Perceived Accuracy on Decision Making in Predictive Process Monitoring](https://mlsanity.com/paper/2506.16617/) (2025)
- [Revisiting Predictive Process Monitoring in the Age of Foundation Models: A Comparative Study of Sequence, Tabular, and LLM Approaches](https://mlsanity.com/paper/2607.27797/) (2026)

## Abstract (arXiv)

In this paper, we address the "black-box" problem in predictive process analytics by building interpretable models that are capable to inform both what and why is a prediction. Predictive process analytics is a newly emerged discipline dedicated to providing business process intelligence in modern organisations. It uses event logs, which capture process execution traces in the form of multi-dimensional sequence data, as the key input to train predictive models. These predictive models, often built upon deep learning techniques, can be used to make predictions about the future states of business process execution. We apply attention mechanism to achieve model interpretability. We propose i) two types of attentions: event attention to capture the impact of specific process events on a prediction, and attribute attention to reveal which attribute(s) of an event influenced the prediction; and ii) two attention mechanisms: shared attention mechanism and specialised attention mechanism to reflect different design decisions in when to construct attribute attention on individual input features (specialised) or using the concatenated feature tensor of all input feature vectors (shared). These lead to two distinct attention-based models, and both are interpretable models that incorporate interpretability directly into the structure of a process predictive model. We conduct experimental evaluation of the proposed models using real-life dataset, and comparative analysis between the models for accuracy and interpretability, and draw insights from the evaluation and analysis results.