# Learning optimal policies from event logs through reinforcement learning: a comparison of deep and MDP-based approaches

arXiv: https://arxiv.org/abs/2303.09209 · HTML: https://mlsanity.com/paper/2303.09209/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 42 references that are in the map.
- First posted on arXiv 16 Mar 2023.

## Builds on (most cited first, 12 of 12)

- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Deep Reinforcement Learning for Dialogue Generation](https://mlsanity.com/paper/1606.01541/) (2016)
- [A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems](https://mlsanity.com/paper/2203.01387/) (2022)
- [Predictive Process Monitoring Methods: Which One Suits Me Best?](https://mlsanity.com/paper/1804.02422/) (2018)
- [Prescriptive Business Process Monitoring for Recommending Next Best Actions](https://mlsanity.com/paper/2008.08693/) (2020)
- [Fire Now, Fire Later: Alarm-Based Systems for Prescriptive Process Monitoring](https://mlsanity.com/paper/1905.09568/) (2019)
- [Alarm-Based Prescriptive Process Monitoring](https://mlsanity.com/paper/1803.08706/) (2018)
- [Prescriptive Process Monitoring: Quo Vadis?](https://mlsanity.com/paper/2112.01769/) (2021)
- [Prescriptive Process Monitoring Under Resource Constraints: A Causal Inference Approach](https://mlsanity.com/paper/2109.02894/) (2021)
- [What if Process Predictions are not followed by Good Recommendations? (Technical Report)](https://mlsanity.com/paper/1905.10173/) (2019)
- [Learning to act: a Reinforcement Learning approach to recommend the best next activities](https://mlsanity.com/paper/2203.15398/) (2022)

## Built on by (most cited first, 5 of 5)

- [Automatically Reconciling the Trade-off between Prediction Accuracy and Earliness in Prescriptive Business Process Monitoring](https://mlsanity.com/paper/2307.05939/) (2023)
- [SimBank: from Simulation to Solution in Prescriptive Process Monitoring](https://mlsanity.com/paper/2506.14772/) (2025)
- [ProCause: Generating Counterfactual Outcomes to Evaluate Prescriptive Process Monitoring Methods](https://mlsanity.com/paper/2509.00797/) (2025)
- [Using Reinforcement Learning to Optimize Responses in Care Processes: A Case Study on Aggression Incidents](https://mlsanity.com/paper/2310.00981/) (2023)
- [SCOPE: Sequential Causal Optimization of Process Interventions](https://mlsanity.com/paper/2512.17629/) (2025)

## Abstract (arXiv)

Prescriptive Process Monitoring is an emerging area within Process Mining that focuses on recommending actions to optimize business outcomes. Most existing works prescribe pre-defined interventions, i.e., sets of actions applied to ongoing process executions to achieve a specific objective or Key Performance Indicator (KPI). In contrast, only a few approaches have explored learning and evaluating optimal behavioral policies, i.e., general strategies that determine the best sequence of actions to maximize a desired KPI. In this paper, we address the problem of learning optimal behavioral policies by proposing an AI-based approach that learns an optimal policy directly from historical process executions using Reinforcement Learning (RL) to recommend the best actions for optimizing a KPI. To this end, we employ two RL techniques. The first is a classical model-based approach that extends previous work by the authors through the construction of a Markov Decision Process (MDP) capturing process behavior. The second is a model-free technique based on offline Deep RL. Unlike state-of-the-art work, we aim to minimize the use of domain knowledge and learn optimal policies directly from historical event data. This allows us to learn when to apply interventions and discover effective ones directly from data. Moreover, we target complex scenarios involving external actors, where the process owner controls only part of the activities. We adopt a data-driven Business Process Simulation (BPS) environment to evaluate the learned policies. Results show that both methods improve the targeted KPI with similar effectiveness, while the model-based approach outperforms offline Deep RL in computational efficiency.