mlsanityData to 7 Oct 2026

Evaluation of ChatGPT for NLP-based Mental Health Applications

arXiv 2303.15727 ↗First posted 28 Mar 2023cs.CL cs.AIcited by 53

Not on the map. Why?

Abstract (arXiv)

Large language models (LLM) have been successful in several natural language understanding tasks and could be relevant for natural language processing (NLP)-based mental health application research. In this work, we report the performance of LLM-based ChatGPT (with gpt-3.5-turbo backend) in three text-based mental health classification tasks: stress detection (2-class classification), depression detection (2-class classification), and suicidality detection (5-class classification). We obtained annotated social media posts for the three classification tasks from public datasets. Then ChatGPT API classified the social media posts with an input prompt for classification. We obtained F1 scores of 0.73, 0.86, and 0.37 for stress detection, depression detection, and suicidality detection, respectively. A baseline model that always predicted the dominant class resulted in F1 scores of 0.35, 0.60, and 0.19. The zero-shot classification accuracy obtained with ChatGPT indicates a potential use of language models for mental health classification tasks.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Builds on 6 of its 16 references that are in the map. First posted on arXiv 28 Mar 2023.

Builds on

6 of 16 references.

References · 6

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  4. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  5. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  6. 2019Dreaddit: A Reddit Dataset for Stress Analysis in Social Mediacited by 81

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 17; 2024: 19; 2025: 10; 2026: 7.

53 papers.

Sort:

By area:

  1. 2023Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Datacited by 66
  2. 2024The opportunities and risks of large language models in mental healthcited by 54
  3. 2023Towards Interpretable Mental Health Analysis with Large Language Modelscited by 53
  4. 2023Prompt Engineering for Healthcare: Methodologies and Applicationscited by 52
  5. 2024Large Language Models in Mental Health Care: a Scoping Reviewcited by 34
  6. 2023LLM-empowered Chatbots for Psychiatrist and Patient Simulation: Application and Evaluationcited by 33
  7. 2024AI for social science and social science of AI: A Surveycited by 29
  8. 2023A Zero-shot and Few-shot Study of Instruction-Finetuned Large Language Models Applied to Clinical and Biomedical Taskscited by 25
  9. 2023CohortGPT: An Enhanced GPT for Participant Recruitment in Clinical Studycited by 19
  10. 2024Understanding the Therapeutic Relationship between Counselors and Clients in Online Text-based Counseling using LLMscited by 19
  11. 2024Do Large Language Models Align with Core Mental Health Counseling Competencies?cited by 19
  12. 2023Navigating Prompt Complexity for Zero-Shot Classification: A Study of Large Language Models in Computational Social Sciencecited by 18
38 more
  1. 2023Decoding ChatGPT: A Taxonomy of Existing Research, Current Challenges, and Possible Future Directionscited by 18
  2. 2024Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Reviewcited by 17
  3. 2023MindShift: Leveraging Large Language Models for Mental-States-Based Problematic Smartphone Use Interventioncited by 16
  4. 2024Large Language Model for Mental Health: A Systematic Reviewcited by 14
  5. 2025A Systematic Evaluation of LLM Strategies for Mental Health Text Analysis: Fine-tuning vs. Prompt Engineering vs. RAGcited by 14
  6. 2023PsyEval: A Suite of Mental Health Related Tasks for Evaluating Large Language Modelscited by 13
  7. 2023Bias in Emotion Recognition with ChatGPTcited by 11
  8. 2023From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Modelscited by 11
  9. 2024DepressionEmo: A novel dataset for multilabel classification of depression emotionscited by 10
  10. 2023Towards a Psychological Generalist AI: A Survey of Current Applications of Large Language Models and Future Prospectscited by 8
  11. 2024MindGuard: Towards Accessible and Sitgma-free Mental Health First Aid via Edge LLMcited by 7
  12. 2024A Comprehensive Evaluation of Large Language Models on Mental Illnessescited by 7
  13. 2024Underneath the Numbers: Quantitative and Qualitative Gender Fairness in LLMs for Depression Predictioncited by 6
  14. 2025Survey and Experiments on Mental Disorder Detection via Social Media: From Large Language Models and RAG to Agentscited by 6
  15. 2025ProMind-LLM: Proactive Mental Health Care via Causal Reasoning with Sensor Datacited by 6
  16. 2023PromptMTopic: Unsupervised Multimodal Topic Modeling of Memes using Large Language Modelscited by 5
  17. 2024Unveiling and Mitigating Bias in Mental Health Analysis with Large Language Modelscited by 5
  18. 2025Labeling Free-text Data using Language Model Ensemblescited by 5
  19. 2024Automating PTSD Diagnostics in Clinical Interviews: Leveraging Large Language Models for Trauma Assessmentscited by 4
  20. 2024Advancing Depression Detection on Social Media Platforms Through Fine-Tuned Large Language Modelscited by 4
  21. 2025Can LLMs Address Mental Health Questions? A Comparison with Human Therapistscited by 4
  22. 2024Towards Reliable and Empathetic Depression-Diagnosis-Oriented Chatscited by 3
  23. 2025Menta: A Small Language Model for On-Device Mental Health Predictioncited by 3
  24. 2023ConceptPsy:A Benchmark Suite with Conceptual Comprehensiveness in Psychologycited by 2
  25. 2025Somatic in the East, Psychological in the West?: Investigating Clinically-Grounded Cross-Cultural Depression Symptom Expression in LLMscited by 2
  26. 2026Mental-R1: Aligning LLM Reasoning for Mental Health Assessmentcited by 2
  27. 2023Talk2Care: Facilitating Asynchronous Patient-Provider Communication with Large-Language-Modelcited by 1
  28. 2023Conceptualizing Suicidal Behavior: Utilizing Explanations of Predicted Outcomes to Analyze Longitudinal Social Media Datacited by 1
  29. 2024A Dual-Prompting for Interpretable Mental Health Language Modelscited by 1
  30. 2024Adapting Mental Health Prediction Tasks for Cross-lingual Learning via Meta-Training and In-context Learning with Large Language Modelcited by 1
  31. 2026A Comparative Study of Traditional Machine Learning, Deep Learning, and Large Language Models for Mental Health Forecasting using Smartphone Sensing Datacited by 1
  32. 2024Utilizing AI and Social Media Analytics to Discover Adverse Side Effects of GLP-1 Receptor Agonistscited by 0
  33. 2024PALLM: Evaluating and Enhancing PALLiative Care Conversations with Large Language Modelscited by 0
  34. 2025Predicting and Understanding College Student Mental Health with Interpretable Machine Learningcited by 0
  35. 2025Epitome: Pioneering an Experimental Platform for AI-Social Science Integrationcited by 0
  36. 2025Automated Data Enrichment using Confidence-Aware Fine-Grained Debate among Open-Source LLMs for Mental Health and Online Safetycited by 0
  37. 2026CARE: An Explainable Computational Framework for Assessing Client-Perceived Therapeutic Alliance Using Large Language Modelscited by 0
  38. 2026Beyond Semantics: An Evidential Reasoning-Aware Multi-View Learning Framework for Trustworthy Mental Health Predictioncited by 0

50 most cited of 53 All 53 (JSON)