# A Universal Prompting Strategy for Extracting Process Model Information from Natural Language Text using Large Language Models

arXiv: https://arxiv.org/abs/2407.18540 · HTML: https://mlsanity.com/paper/2407.18540/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 34 references that are in the map.
- First posted on arXiv 26 Jul 2024.

## Builds on (most cited first, 12 of 12)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback](https://mlsanity.com/paper/2305.14387/) (2023)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Challenges and Applications of Large Language Models](https://mlsanity.com/paper/2307.10169/) (2023)
- [Template-Based Named Entity Recognition Using BART](https://mlsanity.com/paper/2106.01760/) (2021)
- [Best Practices for Text Annotation with Large Language Models](https://mlsanity.com/paper/2402.05129/) (2024)
- [Anaphora and Coreference Resolution: A Review](https://mlsanity.com/paper/1805.11824/) (2018)
- [Process Modeling With Large Language Models](https://mlsanity.com/paper/2403.07541/) (2024)
- [An Approach for Process Model Extraction By Multi-Grained Text Classification](https://mlsanity.com/paper/1906.02127/) (2019)
- [Beyond Rule-based Named Entity Recognition and Relation Extraction for Process Model Generation from Natural Language Text](https://mlsanity.com/paper/2305.03960/) (2023)

## Built on by (most cited first, 6 of 6)

- [BPMN Assistant: An LLM-Based Approach to Business Process Modeling](https://mlsanity.com/paper/2509.24592/) (2025)
- [Leveraging Machine Learning and Enhanced Parallelism Detection for BPMN Model Generation from Text](https://mlsanity.com/paper/2507.08362/) (2025)
- [Assisted Data Annotation for Business Process Information Extraction from Textual Documents](https://mlsanity.com/paper/2410.01356/) (2024)
- [Large Language Models to Enhance Business Process Modeling: Past, Present, and Future Trends](https://mlsanity.com/paper/2604.14034/) (2026)
- [Language Mutations Sustain the Persistences of Conspiracy Theories on Social Media](https://mlsanity.com/paper/2605.20050/) (2026)
- [AI-PAVE-Br: Leveraging Large Language Models for Enhanced Product Attribute Value Extraction through a Golden Set Approach](https://mlsanity.com/paper/2606.24655/) (2026)

## Abstract (arXiv)

Over the past decade, extensive research efforts have been dedicated to the extraction of information from textual process descriptions. Despite the remarkable progress witnessed in natural language processing (NLP), information extraction within the Business Process Management domain remains predominantly reliant on rule-based systems and machine learning methodologies. Data scarcity has so far prevented the successful application of deep learning techniques. However, the rapid progress in generative large language models (LLMs) makes it possible to solve many NLP tasks with very high quality without the need for extensive data. Therefore, we systematically investigate the potential of LLMs for extracting information from textual process descriptions, targeting the detection of process elements such as activities and actors, and relations between them. Using a heuristic algorithm, we demonstrate the suitability of the extracted information for process model generation. Based on a novel prompting strategy, we show that LLMs are able to outperform state-of-the-art machine learning approaches with absolute performance improvements of up to 8% F1 score across three different datasets. We evaluate our prompting strategy on eight different LLMs, showing it is universally applicable, while also analyzing the impact of certain prompt parts on extraction quality. The number of example texts, the specificity of definitions, and the rigour of format instructions are identified as key for improving the accuracy of extracted information. Our code, prompts, and data are publicly available.