# RACER: An LLM-powered Methodology for Scalable Analysis of Semi-structured Mental Health Interviews

arXiv: https://arxiv.org/abs/2402.02656 · HTML: https://mlsanity.com/paper/2402.02656/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 26 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Builds on (most cited first, 4 of 4)

- [A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models](https://mlsanity.com/paper/2401.01313/) (2024)
- [Quantifying Uncertainty in Natural Language Explanations of Large Language Models](https://mlsanity.com/paper/2311.03533/) (2023)
- [Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare delivery](https://mlsanity.com/paper/2304.13714/) (2023)
- [Harnessing ChatGPT for thematic analysis: Are we ready?](https://mlsanity.com/paper/2310.14545/) (2023)

## Built on by (most cited first, 6 of 6)

- [PsycoLLM: Enhancing LLM for Psychological Understanding and Evaluation](https://mlsanity.com/paper/2407.05721/) (2024)
- [Requirements Elicitation Follow-Up Question Generation](https://mlsanity.com/paper/2507.02858/) (2025)
- [Exploring the Human-LLM Synergy in Advancing Theory-driven Qualitative Analysis](https://mlsanity.com/paper/2405.05758/) (2024)
- [LLM-TA: An LLM-Enhanced Thematic Analysis Pipeline for Transcripts from Parents of Children with Congenital Heart Disease](https://mlsanity.com/paper/2502.01620/) (2025)
- [Word Clouds as Common Voices: LLM-Assisted Visualization of Participant-Weighted Themes in Qualitative Interviews](https://mlsanity.com/paper/2508.07517/) (2025)
- [Can LLMs Understand the Impact of Trauma? Costs and Benefits of LLMs Coding the Interviews of Firearm Violence Survivors](https://mlsanity.com/paper/2604.16132/) (2026)

## Abstract (arXiv)

Semi-structured interviews (SSIs) are a commonly employed data-collection method in healthcare research, offering in-depth qualitative insights into subject experiences. Despite their value, the manual analysis of SSIs is notoriously time-consuming and labor-intensive, in part due to the difficulty of extracting and categorizing emotional responses, and challenges in scaling human evaluation for large populations. In this study, we develop RACER, a Large Language Model (LLM) based expert-guided automated pipeline that efficiently converts raw interview transcripts into insightful domain-relevant themes and sub-themes. We used RACER to analyze SSIs conducted with 93 healthcare professionals and trainees to assess the broad personal and professional mental health impacts of the COVID-19 crisis. RACER achieves moderately high agreement with two human evaluators (72%), which approaches the human inter-rater agreement (77%). Interestingly, LLMs and humans struggle with similar content involving nuanced emotional, ambivalent/dialectical, and psychological statements. Our study highlights the opportunities and challenges in using LLMs to improve research efficiency and opens new avenues for scalable analysis of SSIs in healthcare research.