# Domain Adaptation of LLMs for Process Data

arXiv: https://arxiv.org/abs/2509.03161 · HTML: https://mlsanity.com/paper/2509.03161/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 19 references that are in the map.
- First posted on arXiv 3 Sep 2025.

## Builds on (most cited first, 11 of 11)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [LoRA Learns Less and Forgets Less](https://mlsanity.com/paper/2405.09673/) (2024)
- [Predictive Business Process Monitoring with LSTM Neural Networks](https://mlsanity.com/paper/1612.02130/) (2016)
- [Deep Learning for Predictive Business Process Monitoring: Review and Benchmark](https://mlsanity.com/paper/2009.13251/) (2020)
- [When Do Prompting and Prefix-Tuning Work? A Theory of Capabilities and Limitations](https://mlsanity.com/paper/2310.19698/) (2023)
- [Creating Unbiased Public Benchmark Datasets with Data Leakage Prevention for Predictive Process Monitoring](https://mlsanity.com/paper/2107.01905/) (2021)
- [PM-LLM-Benchmark: Evaluating Large Language Models on Process Mining Tasks](https://mlsanity.com/paper/2407.13244/) (2024)
- [Evaluating the Ability of LLMs to Solve Semantics-Aware Process Mining Tasks](https://mlsanity.com/paper/2407.02310/) (2024)
- [What Averages Do Not Tell -- Predicting Real Life Processes with Sequential Deep Learning](https://mlsanity.com/paper/2110.10225/) (2021)

## Built on by (most cited first, 6 of 6)

- [Time Series Foundation Models for Process Model Forecasting](https://mlsanity.com/paper/2512.07624/) (2025)
- [David vs. Goliath in Next Activity Prediction: Argmax vs. LSTM, Transformer, and LLM](https://mlsanity.com/paper/2606.15868/) (2026)
- [Revisiting Predictive Process Monitoring in the Age of Foundation Models: A Comparative Study of Sequence, Tabular, and LLM Approaches](https://mlsanity.com/paper/2607.27797/) (2026)
- [D-TAIA: Domain-Aware LLM Adaptation for Multi-Task Predictive Process Monitoring](https://mlsanity.com/paper/2608.28236/) (2026)
- [Efficient Online Continual Foundation Model Fine-Tuning for Predictive Process Monitoring](https://mlsanity.com/paper/2608.28237/) (2026)
- [From Event Logs to Governed Action: A BlueSky Agenda for Agentic Process Mining](https://mlsanity.com/paper/2609.07984/) (2026)

## Abstract (arXiv)

In recent years, Large Language Models (LLMs) have emerged as a prominent area of interest across various research domains, including Process Mining (PM). Current applications in PM have predominantly centered on prompt engineering strategies or the transformation of event logs into narrative-style datasets, thereby exploiting the semantic capabilities of LLMs to address diverse tasks. In contrast, this study investigates the direct adaptation of pretrained LLMs to process data without natural language reformulation, motivated by the fact that these models excel in generating sequences of tokens, similar to the objective in PM. More specifically, we focus on parameter-efficient fine-tuning techniques to mitigate the computational overhead typically associated with such models. Our experimental setup focuses on Predictive Process Monitoring (PPM), and considers both single- and multi-task predictions. The results demonstrate a potential improvement in predictive performance over state-of-the-art recurrent neural network (RNN) approaches and recent narrative-style-based solutions, particularly in the multi-task setting. Additionally, our fine-tuned models exhibit faster convergence and require significantly less hyperparameter optimization.