mlsanityData to 14 Oct 2026

Still Not Quite There! Evaluating Large Language Models for Comorbid Mental Health Diagnosis

arXiv 2410.03908 ↗First posted 4 Oct 2024cs.CL cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

In this study, we introduce ANGST, a novel, first-of-its kind benchmark for depression-anxiety comorbidity classification from social media posts. Unlike contemporary datasets that often oversimplify the intricate interplay between different mental health disorders by treating them as isolated conditions, ANGST enables multi-label classification, allowing each post to be simultaneously identified as indicating depression and/or anxiety. Comprising 2876 meticulously annotated posts by expert psychologists and an additional 7667 silver-labeled posts, ANGST posits a more representative sample of online mental health discourse. Moreover, we benchmark ANGST using various state-of-the-art language models, ranging from Mental-BERT to GPT-4. Our results provide significant insights into the capabilities and limitations of these models in complex diagnostic scenarios. While GPT-4 generally outperforms other models, none achieve an F1 score exceeding 72% in multi-class comorbid classification, underscoring the ongoing challenges in applying language models to mental health diagnostics.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 32 of its 83 references that are in the map. First posted on arXiv 4 Oct 2024.

Builds on

32 of 83 references.

References · 32

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  7. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  8. 2020Learning under Concept Drift: A Reviewcited by 477
  9. 2013Crowdsourcing a Word-Emotion Association Lexiconcited by 196
  10. 2021Want To Reduce Labeling Cost? GPT-3 Can Helpcited by 196
  11. 2023AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotatorscited by 172
  12. 2021MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcarecited by 161
20 more
  1. 2017Depression and Self-Harm Risk Assessment in Online Forumscited by 84
  2. 2019Dreaddit: A Reddit Dataset for Stress Analysis in Social Mediacited by 81
  3. 2018SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditionscited by 73
  4. 2023Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Datacited by 66
  5. 2023MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Modelscited by 63
  6. 2016Empath: Understanding Topic Signals in Large-Scale Textcited by 51
  7. 2021Deep Learning for Suicide and Depression Identification with Unsupervised Label Correctioncited by 45
  8. 2023Benefits and Harms of Large Language Models in Digital Mental Healthcited by 40
  9. 2022CAMS: An Annotated Corpus for Causal Analysis of Mental Health Issues in Social Media Postscited by 34
  10. 2019Suicidal Ideation Detection: A Review of Machine Learning Methods and Applicationscited by 31
  11. 2022Improving the Generalizability of Depression Detection by Leveraging Clinical Questionnairescited by 27
  12. 2023Two Directions for Clinical Data Generation with Large Language Models: Data-to-Label and Label-to-Datacited by 26
  13. 2023Domain-specific Continued Pretraining of Language Models for Capturing Long Context in Mental Healthcited by 25
  14. 2022Hierarchical Attention Network for Explainable Depression Detection on Twitter Aided by Metaphor Concept Mappingscited by 23
  15. 2023ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?cited by 15
  16. 2017A Short Review of Ethical Challenges in Clinical Natural Language Processingcited by 14
  17. 2020Towards Preemptive Detection of Depression and Anxiety in Twittercited by 13
  18. 2023Emotion fusion for mental illness detection from social media: A surveycited by 13
  19. 2023Revisiting Hate Speech Benchmarks: From Data Curation to System Deploymentcited by 7
  20. 2023Effective Proxy for Human Labeling: Ensemble Disagreement Scores in Large Language Models for Industrial NLPcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 5.

12 papers.

  1. 2025A Survey of Large Language Models in Psychotherapy: Current Landscape and Future Directionscited by 12
  2. 2025MHQA: A Diverse, Knowledge Intensive Mental Health Question Answering Challenge for Language Modelscited by 7
  3. 2025ProMind-LLM: Proactive Mental Health Care via Causal Reasoning with Sensor Datacited by 6
  4. 2025FedMentor: Domain-Aware Differential Privacy for Heterogeneous Federated LLMs in Mental Healthcited by 5
  5. 2025A Multi-Level Benchmark for Causal Language Understanding in Social Media Discoursecited by 3
  6. 2025ReDepress: A Cognitive Framework for Detecting Depression Relapse from Social Mediacited by 2
  7. 2025It Hears, It Sees too: Multi-Modal LLM for Depression Detection By Integrating Visual Understanding into Audio Language Modelscited by 2
  8. 2026Responsible Evaluation of AI for Mental Healthcited by 2
  9. 2026OMIND: Framework for Knowledge Grounded Finetuning and Multi-Turn Dialogue Benchmark for Mental Health LLMscited by 1
  10. 2026When PCOS Meets Eating Disorders: An Explainable AI Approach to Detecting the Hidden Triple Burdencited by 0
  11. 2026The MADRS Pipeline: Supporting Depression Assessment in Clinical Trialscited by 0
  12. 2026From Pattern Recognizers to Personalized Companions: A Survey of Large Language Models in Mental Healthcited by 0