# What Averages Do Not Tell -- Predicting Real Life Processes with Sequential Deep Learning

arXiv: https://arxiv.org/abs/2110.10225 · HTML: https://mlsanity.com/paper/2110.10225/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 19 Oct 2021.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Multimodal Transformer for Unaligned Multimodal Language Sequences](https://mlsanity.com/paper/1906.00295/) (2019)
- [Tensor Fusion Network for Multimodal Sentiment Analysis](https://mlsanity.com/paper/1707.07250/) (2017)
- [Predictive Business Process Monitoring with LSTM Neural Networks](https://mlsanity.com/paper/1612.02130/) (2016)
- [Predicting Process Behaviour using Deep Learning](https://mlsanity.com/paper/1612.04600/) (2016)
- [ProcessTransformer: Predictive Business Process Monitoring with Transformer Network](https://mlsanity.com/paper/2104.00721/) (2021)
- [A systematic literature review on state-of-the-art deep learning methods for process prediction](https://mlsanity.com/paper/2101.09320/) (2021)
- [Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order](https://mlsanity.com/paper/2004.11579/) (2020)
- [A Deep Adversarial Model for Suffix and Remaining Time Prediction of Event Sequences](https://mlsanity.com/paper/2102.07298/) (2021)
- [Creating Unbiased Public Benchmark Datasets with Data Leakage Prevention for Predictive Process Monitoring](https://mlsanity.com/paper/2107.01905/) (2021)
- [Process Outcome Prediction: CNN vs. LSTM (with Attention)](https://mlsanity.com/paper/2104.06934/) (2021)

## Built on by (most cited first, 7 of 7)

- [Neuro-Symbolic Predictive Process Monitoring](https://mlsanity.com/paper/2509.00834/) (2025)
- [Modeling Events and Interactions through Temporal Processes -- A Survey](https://mlsanity.com/paper/2303.06067/) (2023)
- [Domain Adaptation of LLMs for Process Data](https://mlsanity.com/paper/2509.03161/) (2025)
- [Knowledge-Driven Modulation of Neural Networks with Attention Mechanism for Next Activity Prediction](https://mlsanity.com/paper/2312.08847/) (2023)
- [Encoder-Decoder Model for Suffix Prediction in Predictive Monitoring](https://mlsanity.com/paper/2211.16106/) (2022)
- [An Uncertainty-Aware ED-LSTM for Probabilistic Suffix Prediction](https://mlsanity.com/paper/2505.21339/) (2025)
- [Probabilistic Forecasting of Business Process Executions with Neural Temporal Point Processes](https://mlsanity.com/paper/2609.21382/) (2026)

## Abstract (arXiv)

Deep Learning is proven to be an effective tool for modeling sequential data as shown by the success in Natural Language, Computer Vision and Signal Processing. Process Mining concerns discovering insights on business processes from their execution data that are logged by supporting information systems. The logged data (event log) is formed of event sequences (traces) that correspond to executions of a process. Many Deep Learning techniques have been successfully adapted for predictive Process Mining that aims to predict process outcomes, remaining time, the next event, or even the suffix of running traces. Traces in Process Mining are multimodal sequences and very differently structured than natural language sentences or images. This may require a different approach to processing. So far, there has been little focus on these differences and the challenges introduced. Looking at suffix prediction as the most challenging of these tasks, the performance of Deep Learning models was evaluated only on average measures and for a small number of real-life event logs. Comparing the results between papers is difficult due to different pre-processing and evaluation strategies. Challenges that may be relevant are the skewness of trace-length distribution and the skewness of the activity distribution in real-life event logs. We provide an end-to-end framework which enables to compare the performance of seven state-of-the-art sequential architectures in common settings. Results show that sequence modeling still has a lot of room for improvement for majority of the more complex datasets. Further research and insights are required to get consistent performance not just in average measures but additionally over all the prefixes.