# Multi-Label Classification with Generative AI Models in Healthcare: A Case Study of Suicidality and Risk Factors

arXiv: https://arxiv.org/abs/2507.17009 · HTML: https://mlsanity.com/paper/2507.17009/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 22 Jul 2025.

## Builds on (most cited first, 9 of 9)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [The Emerging Trends of Multi-Label Learning](https://mlsanity.com/paper/2011.11197/) (2020)
- [SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions](https://mlsanity.com/paper/1806.05258/) (2018)
- [Text-to-Table: A New Way of Information Extraction](https://mlsanity.com/paper/2109.02707/) (2021)
- [Comprehensive Comparative Study of Multi-Label Classification Methods](https://mlsanity.com/paper/2102.07113/) (2021)
- [ScAN: Suicide Attempt and Ideation Events Dataset](https://mlsanity.com/paper/2205.07872/) (2022)
- [Suicide Phenotyping from Clinical Notes in Safety-Net Psychiatric Hospital Using Multi-Label Classification with Pre-Trained Language Models](https://mlsanity.com/paper/2409.18878/) (2024)

## Abstract (arXiv)

Suicide remains a pressing global health crisis, with over 720,000 deaths annually and millions more affected by suicide ideation (SI) and suicide attempts (SA). Early identification of suicidality-related factors (SrFs), including SI, SA, exposure to suicide (ES), and non-suicidal self-injury (NSSI), is critical for timely intervention. While prior studies have applied AI to detect SrFs in clinical notes, most treat suicidality as a binary classification task, overlooking the complexity of cooccurring risk factors. This study explores the use of generative large language models (LLMs), specifically GPT-3.5 and GPT-4.5, for multi-label classification (MLC) of SrFs from psychiatric electronic health records (EHRs). We present a novel end to end generative MLC pipeline and introduce advanced evaluation methods, including label set level metrics and a multilabel confusion matrix for error analysis. Finetuned GPT-3.5 achieved top performance with 0.94 partial match accuracy and 0.91 F1 score, while GPT-4.5 with guided prompting showed superior performance across label sets, including rare or minority label sets, indicating a more balanced and robust performance. Our findings reveal systematic error patterns, such as the conflation of SI and SA, and highlight the models tendency toward cautious over labeling. This work not only demonstrates the feasibility of using generative AI for complex clinical classification tasks but also provides a blueprint for structuring unstructured EHR data to support large scale clinical research and evidence based medicine.