# On the Potential of Large Language Models to Solve Semantics-Aware Process Mining Tasks

arXiv: https://arxiv.org/abs/2504.21074 · HTML: https://mlsanity.com/paper/2504.21074/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 46 references that are in the map.
- First posted on arXiv 29 Apr 2025.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://mlsanity.com/paper/2204.07705/) (2022)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [Predicting Process Behaviour using Deep Learning](https://mlsanity.com/paper/1612.04600/) (2016)
- [Large Language Models can accomplish Business Process Management Tasks](https://mlsanity.com/paper/2307.09923/) (2023)
- [A systematic literature review on state-of-the-art deep learning methods for process prediction](https://mlsanity.com/paper/2101.09320/) (2021)
- [Process Modeling With Large Language Models](https://mlsanity.com/paper/2403.07541/) (2024)
- [Abstractions, Scenarios, and Prompt Definitions for Process Mining with LLMs: A Case Study](https://mlsanity.com/paper/2307.02194/) (2023)
- [PM-LLM-Benchmark: Evaluating Large Language Models on Process Mining Tasks](https://mlsanity.com/paper/2407.13244/) (2024)
- [Evaluating the Ability of LLMs to Solve Semantics-Aware Process Mining Tasks](https://mlsanity.com/paper/2407.02310/) (2024)
- [Evaluating Large Language Models on Business Process Modeling: Framework, Benchmark, and Self-Improvement Analysis](https://mlsanity.com/paper/2412.00023/) (2024)
- [SAP Signavio Academic Models: A Large Process Model Dataset](https://mlsanity.com/paper/2208.12223/) (2022)
- [Chit-Chat or Deep Talk: Prompt Engineering for Process Mining](https://mlsanity.com/paper/2307.09909/) (2023)
- [Multivariate Business Process Representation Learning utilizing Gramian Angular Fields and Convolutional Neural Networks](https://mlsanity.com/paper/2106.08027/) (2021)
- [Process Extraction from Text: Benchmarking the State of the Art and Paving the Way for Future Challenges](https://mlsanity.com/paper/2110.03754/) (2021)
- [Large Process Models: A Vision for Business Process Management in the Age of Generative AI](https://mlsanity.com/paper/2309.00900/) (2023)
- [xSemAD: Explainable Semantic Anomaly Detection in Event Logs Using Sequence-to-Sequence Models](https://mlsanity.com/paper/2406.19763/) (2024)
- [LLM4PM: A case study on using Large Language Models for Process Modeling in Enterprise Organizations](https://mlsanity.com/paper/2407.17478/) (2024)
- [Bridging Domain Knowledge and Process Discovery Using Large Language Models](https://mlsanity.com/paper/2408.17316/) (2024)
- [Towards a Benchmark for Large Language Models for Business Process Management Tasks](https://mlsanity.com/paper/2410.03255/) (2024)

## Built on by (most cited first, 2 of 2)

- [LLMs that Understand Processes: Instruction-tuning for Semantics-Aware Process Mining](https://mlsanity.com/paper/2508.16270/) (2025)
- [PMAx: An Agentic Framework for AI-Driven Process Mining](https://mlsanity.com/paper/2603.15351/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have shown to be valuable tools for tackling process mining tasks. Existing studies report on their capability to support various data-driven process analyses and even, to some extent, that they are able to reason about how processes work. This reasoning ability suggests that there is potential for LLMs to tackle semantics-aware process mining tasks, which are tasks that rely on an understanding of the meaning of activities and their relationships. Examples of these include process discovery, where the meaning of activities can indicate their dependency, whereas in anomaly detection the meaning can be used to recognize process behavior that is abnormal. In this paper, we systematically explore the capabilities of LLMs for such tasks. Unlike prior work, which largely evaluates LLMs in their default state, we investigate their utility through both in-context learning and supervised fine-tuning. Concretely, we define five process mining tasks requiring semantic understanding and provide extensive benchmarking datasets for evaluation. Our experiments reveal that while LLMs struggle with challenging process mining tasks when used out of the box or with minimal in-context examples, they achieve strong performance when fine-tuned for these tasks across a broad range of process types and industries.