# NLP4PBM: A Systematic Review on Process Extraction using Natural Language Processing with Rule-based, Machine and Deep Learning Methods

arXiv: https://arxiv.org/abs/2409.13738 · HTML: https://mlsanity.com/paper/2409.13738/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 98 references that are in the map.
- First posted on arXiv 10 Sep 2024.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [Self-collaboration Code Generation via ChatGPT](https://mlsanity.com/paper/2304.07590/) (2023)
- [Synchromesh: Reliable code generation from pre-trained language models](https://mlsanity.com/paper/2201.11227/) (2022)
- [Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks](https://mlsanity.com/paper/1810.09536/) (2018)
- [Investigating Gender Bias in BERT](https://mlsanity.com/paper/2009.05021/) (2020)
- [A Comparison of LSTM and BERT for Small Corpus](https://mlsanity.com/paper/2009.05451/) (2020)
- [SkCoder: A Sketch-based Approach for Automatic Code Generation](https://mlsanity.com/paper/2302.06144/) (2023)
- [Large Language Models can accomplish Business Process Management Tasks](https://mlsanity.com/paper/2307.09923/) (2023)
- [An End-to-end Model for Entity-level Relation Extraction using Multi-instance Learning](https://mlsanity.com/paper/2102.05980/) (2021)
- [Process Modeling With Large Language Models](https://mlsanity.com/paper/2403.07541/) (2024)
- [AceCoder: Utilizing Existing Code to Enhance Code Generation](https://mlsanity.com/paper/2303.17780/) (2023)
- [Large Language Models for Business Process Management: Opportunities and Challenges](https://mlsanity.com/paper/2304.04309/) (2023)
- [Abstractions, Scenarios, and Prompt Definitions for Process Mining with LLMs: A Case Study](https://mlsanity.com/paper/2307.02194/) (2023)
- [Retrieval-Augmented Generative Question Answering for Event Argument Extraction](https://mlsanity.com/paper/2211.07067/) (2022)
- [Just Tell Me: Prompt Engineering in Business Process Management](https://mlsanity.com/paper/2304.07183/) (2023)
- [An Approach for Process Model Extraction By Multi-Grained Text Classification](https://mlsanity.com/paper/1906.02127/) (2019)
- [PET: An Annotated Dataset for Process Extraction from Natural Language Text](https://mlsanity.com/paper/2203.04860/) (2022)
- [Beyond Rule-based Named Entity Recognition and Relation Extraction for Process Model Generation from Natural Language Text](https://mlsanity.com/paper/2305.03960/) (2023)
- [Leveraging Data Augmentation for Process Information Extraction](https://mlsanity.com/paper/2404.07501/) (2024)

## Built on by (most cited first, 3 of 3)

- [Knowledge-Driven Hallucination in Large Language Models: An Empirical Study on Process Modeling](https://mlsanity.com/paper/2509.15336/) (2025)
- [Ambiguity Detection and Elimination in Automated Executable Process Modeling](https://mlsanity.com/paper/2604.10884/) (2026)
- [A Hybrid LLM-Based Framework for Automated Security Annotation Generation in Business Process Models](https://mlsanity.com/paper/2608.14370/) (2026)

## Abstract (arXiv)

This literature review studies the field of automated process extraction, i.e., transforming textual descriptions into structured processes using Natural Language Processing (NLP). We found that Machine Learning (ML) / Deep Learning (DL) methods are being increasingly used for the NLP component. In some cases, they were chosen for their suitability towards process extraction, and results show that they can outperform classic rule-based methods. We also found a paucity of gold-standard, scalable annotated datasets, which currently hinders objective evaluations as well as the training or fine-tuning of ML / DL methods. Finally, we discuss preliminary work on the application of LLMs for automated process extraction, as well as promising developments in this field.