# LLM-Orchestrated Conformance Checking in Stroke Care Without Computer-Interpretable Guidelines

arXiv: https://arxiv.org/abs/2606.09489 · HTML: https://mlsanity.com/paper/2606.09489/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 42 references that are in the map.
- First posted on arXiv 8 Jun 2026.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Natural Language Processing: State of The Art, Current Trends and Challenges](https://mlsanity.com/paper/1708.05148/) (2017)
- [Advancing Multimodal Medical Capabilities of Gemini](https://mlsanity.com/paper/2405.03162/) (2024)
- [Large Language Models can accomplish Business Process Management Tasks](https://mlsanity.com/paper/2307.09923/) (2023)
- [Abstractions, Scenarios, and Prompt Definitions for Process Mining with LLMs: A Case Study](https://mlsanity.com/paper/2307.02194/) (2023)
- [ProMoAI: Process Modeling with Generative AI](https://mlsanity.com/paper/2403.04327/) (2024)
- [Fairness-Aware Process Mining](https://mlsanity.com/paper/1908.11451/) (2019)
- [Chit-Chat or Deep Talk: Prompt Engineering for Process Mining](https://mlsanity.com/paper/2307.09909/) (2023)
- [Conformance checking: A state-of-the-art literature review](https://mlsanity.com/paper/2007.10903/) (2020)
- [Leveraging Large Language Models for Enhanced Process Model Comprehension](https://mlsanity.com/paper/2408.08892/) (2024)
- [HealthBranches: Synthesizing Clinically-Grounded Question Answering Datasets via Decision Pathways](https://mlsanity.com/paper/2508.07308/) (2025)
- [Orchid: Orchestrating Context Across Creative Workflows with Generative AI](https://mlsanity.com/paper/2508.19517/) (2025)

## Abstract (arXiv)

Objective: Conformance checking in healthcare seeks to assess whether patient care pathways adhere to clinical guidelines. However, its practical application often depends on the availability of formal, machine-interpretable representations of guidelines, such as Computer-Interpretable Guidelines (CIGs), which are seldom available in real-world clinical settings. Methods: This work introduces a modular framework based on the orchestration of Large Language Models (LLMs) to support medical conformance checking directly from unstructured clinical and guideline texts, without requiring predefined CIGs. The proposed architecture integrates multiple LLMs and supporting components to extract patient traces from clinical discharge letters, identify normative rules from textual clinical guidelines, translate these rules into executable scripts, and compute a Trace Conformance Indicator to quantify compliance within the event log. Results: The framework was implemented and evaluated in the stroke care domain at the neurological ward of Alessandria Hospital. Hundreds of patient traces were automatically extracted from hospital data and assessed against 50 rules derived from the reference guideline. The analysis showed that more than 86% of the available traces were conformant. Conclusion: The results demonstrate the feasibility of using orchestrated LLMs for practical healthcare conformance analysis. At the same time, the study provides evidence of a high level of adherence to stroke care guidelines at Alessandria Hospital.