Automated Healthcare Thematic Analysis using Multi-Agent Large Language Model: Algorithm Development and Evaluation
On the map: LLM agents and tool use → Medical and scientific domain agents
Abstract (arXiv)
Understanding patients experiences is essential for advancing patient-centered care. Qualitative thematic analysis is widely used to explore these experiences, however, the process remains labor-intensive, subjective, and difficult to scale. This study aimed to develop and evaluate Collaborative Theme Identification Agent (CoTI), a multi-agent large language model framework designed to support manual thematic analysis by rapidly generating supporting excerpts, initial codes, and themes. CoTI consists of three agents: Instructor, Thematizer, and CodebookGenerator. The Instructor refines instruction prompts, the Thematizer extracts supporting excerpts and generates initial codes for each transcript, and the CodebookGenerator groups similar codes across all transcripts into a codebook with themes. We evaluated CoTI primarily using 12 heart failure patient transcripts. CoTI-generated outputs were compared against the reference standard developed by senior investigators. To explore human-AI interaction in thematic analysis, we further implemented CoTI in a user-facing application. CoTI generated supporting excerpts, initial codes, and themes that were more similar to those of senior investigators than did the outputs of junior investigators, baseline natural language processing models, and other basic large language models. In an exploratory human-AI collaboration experiment, we found that the collaboration between CoTI and junior investigators provided only marginal gains compared to CoTI alone. A possible hypothesis was that junior investigators may over-rely on CoTI and limit their independent critical thinking. CoTI can improve the efficiency of thematic analysis by rapidly generating supporting excerpts, initial codes, and themes for human researchers review. These findings highlight CoTI potential as a useful tool for scalable qualitative research.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 8 of its 24 references that are in the map. First posted on arXiv 18 Dec 2025.
Builds on
8 of 24 references.
Closest ancestors · 2
Same direction, newest first
- 2025Auto-TA: Towards Scalable Automated Thematic Analysis (TA) via Multi-Agent Large Language Models with Reinforcement Learning30 Jun
- 2025TAMA: A Human-AI Collaborative Thematic Analysis Framework Using Multi-Agent LLMs for Clinical Interviews26 Mar
Other references · 6
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 779
- 2024TextGrad: Automatic "Differentiation" via Textcited by 250
- 2020Top2Vec: Distributed Representations of Topicscited by 124
- 2024A Framework for Human Evaluation of Large Language Models in Healthcare Derived from Literature Reviewcited by 65
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025An Agentic AI System for Multi-Framework Communication Coding
- 2025An Agentic AI Workflow for Detecting Cognitive Concerns in Real-world Data
- 2025Automated Clinical Problem Detection from SOAP Notes using a Collaborative Multi-Agent LLM Architecture
- 2025SFT-TA: Supervised Fine-Tuned Agents in Multi-Agent LLMs for Automated Inductive Thematic Analysis
- 2024Can Large Language Models Serve as Data Analysts? A Multi-Agent Assisted Approach for Qualitative Data Analysis
- 2026SymptomAI: Toward a Conversational AI Agent for Everyday Symptom Assessment
- 2026Retrieval-Augmented Agentic Rubric Generation for Reliable Medical Response Evaluation
- 2026Evaluating Scaffolding-Oriented Multi-Agent Large Language Model System for Clinical Interview Training