# ProcessTBench: An LLM Plan Generation Dataset for Process Mining

arXiv: https://arxiv.org/abs/2409.09191 · HTML: https://mlsanity.com/paper/2409.09191/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 6 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Builds on (most cited first, 4 of 4)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [TaskBench: Benchmarking Large Language Models for Task Automation](https://mlsanity.com/paper/2311.18760/) (2023)
- [On the Tool Manipulation Capability of Open-source Large Language Models](https://mlsanity.com/paper/2305.16504/) (2023)

## Built on by (most cited first, 2 of 2)

- [Skill Learning Using Process Mining for Large Language Model Plan Generation](https://mlsanity.com/paper/2410.12870/) (2024)
- [Automated Event Log Generation from Unstructured Text Using Finetuned LLMs](https://mlsanity.com/paper/2609.01320/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have shown significant promise in plan generation. Yet, existing datasets often lack the complexity needed for advanced tool use scenarios - such as handling paraphrased query statements, supporting multiple languages, and managing actions that can be done in parallel. These scenarios are crucial for evaluating the evolving capabilities of LLMs in real-world applications. Moreover, current datasets don't enable the study of LLMs from a process perspective, particularly in scenarios where understanding typical behaviors and challenges in executing the same process under different conditions or formulations is crucial. To address these gaps, we present the ProcessTBench synthetic dataset, an extension of the TaskBench dataset specifically designed to evaluate LLMs within a process mining framework.