# Creating Unbiased Public Benchmark Datasets with Data Leakage Prevention for Predictive Process Monitoring

arXiv: https://arxiv.org/abs/2107.01905 · HTML: https://mlsanity.com/paper/2107.01905/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 8 references that are in the map.
- First posted on arXiv 5 Jul 2021.

## Builds on (most cited first, 5 of 5)

- [Predictive Business Process Monitoring with LSTM Neural Networks](https://mlsanity.com/paper/1612.02130/) (2016)
- [Predicting Process Behaviour using Deep Learning](https://mlsanity.com/paper/1612.04600/) (2016)
- [Survey and cross-benchmark comparison of remaining time prediction methods in business process monitoring](https://mlsanity.com/paper/1805.02896/) (2018)
- [Outcome-Oriented Predictive Process Monitoring: Review and Benchmark](https://mlsanity.com/paper/1707.06766/) (2017)
- [A systematic literature review on state-of-the-art deep learning methods for process prediction](https://mlsanity.com/paper/2101.09320/) (2021)

## Built on by (most cited first, 16 of 16)

- [PGTNet: A Process Graph Transformer Network for Remaining Time Prediction of Business Process Instances](https://mlsanity.com/paper/2404.06267/) (2024)
- [What Averages Do Not Tell -- Predicting Real Life Processes with Sequential Deep Learning](https://mlsanity.com/paper/2110.10225/) (2021)
- [Domain Adaptation of LLMs for Process Data](https://mlsanity.com/paper/2509.03161/) (2025)
- [Machine learning in business process management: A systematic literature review](https://mlsanity.com/paper/2405.16396/) (2024)
- [Achieving Group Fairness through Independence in Predictive Process Monitoring](https://mlsanity.com/paper/2412.04914/) (2024)
- [A Discussion on Generalization in Next-Activity Prediction](https://mlsanity.com/paper/2309.09618/) (2023)
- [Learning Uncertainty with Artificial Neural Networks for Improved Predictive Process Monitoring](https://mlsanity.com/paper/2206.06317/) (2022)
- [Explainable Artificial Intelligence for Improved Modeling of Processes](https://mlsanity.com/paper/2212.00695/) (2022)
- [Nirdizati: an Advanced Predictive Process Monitoring Toolkit](https://mlsanity.com/paper/2210.09688/) (2022)
- [DIFF-ERO: A Conformance-Aware Loss for Deep Learning in Process Mining](https://mlsanity.com/paper/2606.14283/) (2026)
- [CoSMo: a Framework to Instantiate Conditioned Process Simulation Models](https://mlsanity.com/paper/2303.17879/) (2023)
- [Exploring LLM Features in Predictive Process Monitoring for Small-Scale Event-Logs](https://mlsanity.com/paper/2601.11468/) (2026)
- [David vs. Goliath in Next Activity Prediction: Argmax vs. LSTM, Transformer, and LLM](https://mlsanity.com/paper/2606.15868/) (2026)
- [Revisiting Predictive Process Monitoring in the Age of Foundation Models: A Comparative Study of Sequence, Tabular, and LLM Approaches](https://mlsanity.com/paper/2607.27797/) (2026)
- [On the Potential of Multi-Task Learning in Predictive Process Monitoring](https://mlsanity.com/paper/2609.13477/) (2026)
- [Probabilistic Forecasting of Business Process Executions with Neural Temporal Point Processes](https://mlsanity.com/paper/2609.21382/) (2026)

## Abstract (arXiv)

Advances in AI, and especially machine learning, are increasingly drawing research interest and efforts towards predictive process monitoring, the subfield of process mining (PM) that concerns predicting next events, process outcomes and remaining execution times. Unfortunately, researchers use a variety of datasets and ways to split them into training and test sets. The documentation of these preprocessing steps is not always complete. Consequently, research results are hard or even impossible to reproduce and to compare between papers. At times, the use of non-public domain knowledge further hampers the fair competition of ideas. Often the training and test sets are not completely separated, a data leakage problem particular to predictive process monitoring. Moreover, test sets usually suffer from bias in terms of both the mix of case durations and the number of running cases. These obstacles pose a challenge to the field's progress. The contribution of this paper is to identify and demonstrate the importance of these obstacles and to propose preprocessing steps to arrive at unbiased benchmark datasets in a principled way, thus creating representative test sets without data leakage with the aim of levelling the playing field, promoting open science and contributing to more rapid progress in predictive process monitoring.