# Can recurrent neural networks learn process model structure?

arXiv: https://arxiv.org/abs/2212.06430 · HTML: https://mlsanity.com/paper/2212.06430/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 39 references that are in the map.
- First posted on arXiv 13 Dec 2022.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Predictive Business Process Monitoring with LSTM Neural Networks](https://mlsanity.com/paper/1612.02130/) (2016)
- [Predicting Process Behaviour using Deep Learning](https://mlsanity.com/paper/1612.04600/) (2016)
- [ProcessTransformer: Predictive Business Process Monitoring with Transformer Network](https://mlsanity.com/paper/2104.00721/) (2021)
- [Recurrent Batch Normalization](https://mlsanity.com/paper/1603.09025/) (2016)
- [Process Mining for Python (PM4Py): Bridging the Gap Between Process- and Data Science](https://mlsanity.com/paper/1905.06169/) (2019)
- [Predictive Business Process Monitoring via Generative Adversarial Nets: The Case of Next Event Prediction](https://mlsanity.com/paper/2003.11268/) (2020)
- [Evaluating the Ability of LSTMs to Learn Context-Free Grammars](https://mlsanity.com/paper/1811.02611/) (2018)
- [An Interdisciplinary Comparison of Sequence Modeling Methods for Next-Element Prediction](https://mlsanity.com/paper/1811.00062/) (2018)
- [Embedding Graph Convolutional Networks in Recurrent Neural Networks for Predictive Monitoring](https://mlsanity.com/paper/2112.09641/) (2021)
- [An empirical comparison of deep-neural-network architectures for next activity prediction using context-enriched process event logs](https://mlsanity.com/paper/2005.01194/) (2020)
- [Can deep neural networks learn process model structure? An assessment framework and analysis](https://mlsanity.com/paper/2202.11985/) (2022)

## Built on by (most cited first, 3 of 3)

- [A Discussion on Generalization in Next-Activity Prediction](https://mlsanity.com/paper/2309.09618/) (2023)
- [Recent Advances in Data-Driven Business Process Management](https://mlsanity.com/paper/2406.01786/) (2024)
- [DIFF-ERO: A Conformance-Aware Loss for Deep Learning in Process Mining](https://mlsanity.com/paper/2606.14283/) (2026)

## Abstract (arXiv)

Various methods using machine and deep learning have been proposed to tackle different tasks in predictive process monitoring, forecasting for an ongoing case e.g. the most likely next event or suffix, its remaining time, or an outcome-related variable. Recurrent neural networks (RNNs), and more specifically long short-term memory nets (LSTMs), stand out in terms of popularity. In this work, we investigate the capabilities of such an LSTM to actually learn the underlying process model structure of an event log. We introduce an evaluation framework that combines variant-based resampling and custom metrics for fitness, precision and generalization. We evaluate 4 hypotheses concerning the learning capabilities of LSTMs, the effect of overfitting countermeasures, the level of incompleteness in the training set and the level of parallelism in the underlying process model. We confirm that LSTMs can struggle to learn process model structure, even with simplistic process data and in a very lenient setup. Taking the correct anti-overfitting measures can alleviate the problem. However, these measures did not present themselves to be optimal when selecting hyperparameters purely on predicting accuracy. We also found that decreasing the amount of information seen by the LSTM during training, causes a sharp drop in generalization and precision scores. In our experiments, we could not identify a relationship between the extent of parallelism in the model and the generalization capability, but they do indicate that the process' complexity might have impact.