# Automated Thematic Analyses Using LLMs: Xylazine Wound Management Social Media Chatter Use Case

arXiv: https://arxiv.org/abs/2507.10803 · HTML: https://mlsanity.com/paper/2507.10803/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 21 references that are in the map.
- First posted on arXiv 14 Jul 2025.

## Builds on (most cited first, 1 of 1)

- [Evaluation of ChatGPT Family of Models for Biomedical Reasoning and Classification](https://mlsanity.com/paper/2304.02496/) (2023)

## Built on by (most cited first, 1 of 1)

- [Position: Thematic Analysis of Unstructured Clinical Transcripts with Large Language Models](https://mlsanity.com/paper/2509.14597/) (2025)

## Abstract (arXiv)

Background Large language models (LLMs) face challenges in inductive thematic analysis, a task requiring deep interpretive and domain-specific expertise. We evaluated the feasibility of using LLMs to replicate expert-driven thematic analysis of social media data. Methods Using two temporally non-intersecting Reddit datasets on xylazine (n=286 and n=686, for model optimization and validation, respectively) with twelve expert-derived themes, we evaluated five LLMs against expert coding. We modeled the task as a series of binary classifications, rather than a single, multi-label classification, employing zero-, single-, and few-shot prompting strategies and measuring performance via accuracy, precision, recall, and F1-score. Results On the validation set, GPT-4o with two-shot prompting performed best (accuracy: 90.9%; F1-score: 0.71). For high-prevalence themes, model-derived thematic distributions closely mirrored expert classifications (e.g., xylazine use: 13.6% vs. 17.8%; MOUD use: 16.5% vs. 17.8%). Conclusions Our findings suggest that few-shot LLM-based approaches can automate thematic analyses, offering a scalable supplement for qualitative research. Keywords: thematic analysis, large language models, natural language processing, qualitative analysis, social media, prompt engineering, public health