# Human-Centered Evaluation of an LLM-Based Process Modeling Copilot: A Mixed-Methods Study with Domain Experts

arXiv: https://arxiv.org/abs/2603.12895 · HTML: https://mlsanity.com/paper/2603.12895/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 28 references that are in the map.
- First posted on arXiv 13 Mar 2026.

## Builds on (most cited first, 2 of 2)

- [Human-Centered Explainable AI (XAI): From Algorithms to User Experiences](https://mlsanity.com/paper/2110.10790/) (2021)
- [Evaluating Large Language Models on Business Process Modeling: Framework, Benchmark, and Self-Improvement Analysis](https://mlsanity.com/paper/2412.00023/) (2024)

## Built on by (most cited first, 1 of 1)

- [Improving LLM-Generated Process Model Quality Through Reinforcement Learning: The Role of Reward Function Design](https://mlsanity.com/paper/2607.06175/) (2026)

## Abstract (arXiv)

Integrating Large Language Models (LLMs) into business process management tools promises to democratize Business Process Model and Notation (BPMN) modeling for non-experts. While automated frameworks assess syntactic and semantic quality, they miss human factors like trust, usability, and professional alignment. We conducted a mixed-methods evaluation of our proposed solution, an LLM-powered BPMN copilot, with five process modeling experts using focus groups and standardized questionnaires. Our findings reveal a critical tension between acceptable perceived usability (mean CUQ score: 67.2/100) and notably lower trust (mean score: 48.8%), with reliability rated as the most critical concern (M=1.8/5). Furthermore, we identified output-quality issues, prompting difficulties, and a need for the LLM to ask more in-depth clarifying questions about the process. We envision five use cases ranging from domain-expert support to enterprise quality assurance. We demonstrate the necessity of human-centered evaluation complementing automated benchmarking for LLM modeling agents.