# A Systematic Evaluation of LLM Strategies for Mental Health Text Analysis: Fine-tuning vs. Prompt Engineering vs. RAG

arXiv: https://arxiv.org/abs/2503.24307 · HTML: https://mlsanity.com/paper/2503.24307/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Medical and biomedical RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 12 of its 27 references that are in the map.
- First posted on arXiv 31 Mar 2025.

## Placed in

- Retrieval-augmented generation / [Medical and biomedical RAG](https://mlsanity.com/rag/medical-and-biomedical-rag/)

## Builds on (most cited first, 12 of 12)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics](https://mlsanity.com/paper/2310.05694/) (2023)
- [Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Coding](https://mlsanity.com/paper/2304.10548/) (2023)
- [Evaluation of ChatGPT for NLP-based Mental Health Applications](https://mlsanity.com/paper/2303.15727/) (2023)
- [Suicidal Ideation and Mental Disorder Detection with Attentive Relation Networks](https://mlsanity.com/paper/2004.07601/) (2020)
- [Scalable Qualitative Coding with LLMs: Chain-of-Thought Reasoning Matches Human Performance in Some Hermeneutic Tasks](https://mlsanity.com/paper/2401.15170/) (2024)

## Built on by (most cited first, 15 of 15)

- [Trustworthy AI Psychotherapy: Multi-Agent LLM Workflow for Counseling and Explainable Mental Disorder Diagnosis](https://mlsanity.com/paper/2508.11398/) (2025)
- [Survey and Experiments on Mental Disorder Detection via Social Media: From Large Language Models and RAG to Agents](https://mlsanity.com/paper/2504.02800/) (2025)
- [Sanitizing Manufacturing Dataset Labels Using Vision-Language Models](https://mlsanity.com/paper/2506.23465/) (2025)
- [Rule2Text: Natural Language Explanation of Logical Rules in Knowledge Graphs](https://mlsanity.com/paper/2507.23740/) (2025)
- [Confidence-Credibility Aware Weighted Ensembles of Small LLMs Outperform Large LLMs in Emotion Detection](https://mlsanity.com/paper/2512.17630/) (2025)
- [A Novel Multi-Timescale Stability-Preserving Hierarchical Reinforcement Learning Controller Framework for Adaptive Control in High-Dimensional Dynamical Systems](https://mlsanity.com/paper/2510.22420/) (2025)
- [Vision-Language Models for Infrared Industrial Sensing in Additive Manufacturing Scene Description](https://mlsanity.com/paper/2512.11098/) (2025)
- [CMHL: Contrastive Multi-Head Learning for Emotionally Consistent Text Classification](https://mlsanity.com/paper/2603.14078/) (2026)
- [From Baselines to Preferences: A Comparative Study of LoRA/QLoRA and Preference Optimization for Mental Health Text Classification](https://mlsanity.com/paper/2604.00773/) (2026)
- [CAMO: A Class-Aware Minority-Optimized Ensemble for Robust Language Model Evaluation on Imbalanced Data](https://mlsanity.com/paper/2604.07583/) (2026)
- [DreamerNLplus: Interpretable Modeling of Mental Health Dynamics from Social Media Timelines using Hybrid Rule-Based and RAG Methods](https://mlsanity.com/paper/2605.23052/) (2026)
- [Bag of Tricks or Bag of Myths? Reducing Modeling Complexity with Task Knowledge in Explainable Suicide Risk Assessment](https://mlsanity.com/paper/2609.07766/) (2026)
- [Reading Anxiety or Reading the Label? Comparing Fine-Tuned and Frontier Models for Anxiety Detection on Social Media](https://mlsanity.com/paper/2609.20847/) (2026)
- [Aligning with Lived Experience: Heterogeneous Benefits of Fine Tuning in Mental Health Support Generation](https://mlsanity.com/paper/2609.21075/) (2026)
- [From Pattern Recognizers to Personalized Companions: A Survey of Large Language Models in Mental Health](https://mlsanity.com/paper/2609.25186/) (2026)

## Abstract (arXiv)

This study presents a systematic comparison of three approaches for the analysis of mental health text using large language models (LLMs): prompt engineering, retrieval augmented generation (RAG), and fine-tuning. Using LLaMA 3, we evaluate these approaches on emotion classification and mental health condition detection tasks across two datasets. Fine-tuning achieves the highest accuracy (91% for emotion classification, 80% for mental health conditions) but requires substantial computational resources and large training sets, while prompt engineering and RAG offer more flexible deployment with moderate performance (40-68% accuracy). Our findings provide practical insights for implementing LLM-based solutions in mental health applications, highlighting the trade-offs between accuracy, computational requirements, and deployment flexibility.