# From Staff Messages to Actionable Insights: A Multi-Stage LLM Classification Framework for Healthcare Analytics

arXiv: https://arxiv.org/abs/2509.05484 · HTML: https://mlsanity.com/paper/2509.05484/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 17 references that are in the map.
- First posted on arXiv 5 Sep 2025.

## Builds on (most cited first, 5 of 5)

- [A Review of Challenges and Opportunities in Machine Learning for Health](https://mlsanity.com/paper/1806.00388/) (2018)
- [Natural Language Processing in Electronic Health Records in Relation to Healthcare Decision-making: A Systematic Review](https://mlsanity.com/paper/2306.12834/) (2023)
- [Comparative Analysis of Text Classification Approaches in Electronic Health Records](https://mlsanity.com/paper/2005.06624/) (2020)
- [Large Language Models for Patient Comments Multi-Label Classification](https://mlsanity.com/paper/2410.23528/) (2024)
- [A Framework for Evaluating Dashboards in Healthcare](https://mlsanity.com/paper/2009.04792/) (2020)

## Abstract (arXiv)

Hospital call centers serve as the primary contact point for patients within a hospital system. They also generate substantial volumes of staff messages as navigators process patient requests and communicate with the hospital offices following the established protocol restrictions and guidelines. This continuously accumulated large amount of text data can be mined and processed to retrieve insights; however, traditional supervised learning approaches require annotated data, extensive training, and model tuning. Large Language Models (LLMs) offer a paradigm shift toward more computationally efficient methodologies for healthcare analytics. This paper presents a multi-stage LLM-based framework that identifies staff message topics and classifies messages by their reasons in a multi-class fashion. In the process, multiple LLM types, including reasoning, general-purpose, and lightweight models, were evaluated. The best-performing model was o3, achieving 78.4% weighted F1-score and 79.2% accuracy, followed closely by gpt-5 (75.3% Weighted F1-score and 76.2% accuracy). The proposed methodology incorporates data security measures and HIPAA compliance requirements essential for healthcare environments. The processed LLM outputs are integrated into a visualization decision support tool that transforms the staff messages into actionable insights accessible to healthcare professionals. This approach enables more efficient utilization of the collected staff messaging data, identifies navigator training opportunities, and supports improved patient experience and care quality.