# Still Not Quite There! Evaluating Large Language Models for Comorbid Mental Health Diagnosis

arXiv: https://arxiv.org/abs/2410.03908 · HTML: https://mlsanity.com/paper/2410.03908/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 83 references that are in the map.
- First posted on arXiv 4 Oct 2024.

## Builds on (most cited first, 30 of 32)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [Crowdsourcing a Word-Emotion Association Lexicon](https://mlsanity.com/paper/1308.6297/) (2013)
- [Want To Reduce Labeling Cost? GPT-3 Can Help](https://mlsanity.com/paper/2108.13487/) (2021)
- [AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators](https://mlsanity.com/paper/2303.16854/) (2023)
- [MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare](https://mlsanity.com/paper/2110.15621/) (2021)
- [Depression and Self-Harm Risk Assessment in Online Forums](https://mlsanity.com/paper/1709.01848/) (2017)
- [Dreaddit: A Reddit Dataset for Stress Analysis in Social Media](https://mlsanity.com/paper/1911.00133/) (2019)
- [SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions](https://mlsanity.com/paper/1806.05258/) (2018)
- [Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Data](https://mlsanity.com/paper/2307.14385/) (2023)
- [MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Models](https://mlsanity.com/paper/2309.13567/) (2023)
- [Empath: Understanding Topic Signals in Large-Scale Text](https://mlsanity.com/paper/1602.06979/) (2016)
- [Deep Learning for Suicide and Depression Identification with Unsupervised Label Correction](https://mlsanity.com/paper/2102.09427/) (2021)
- [Benefits and Harms of Large Language Models in Digital Mental Health](https://mlsanity.com/paper/2311.14693/) (2023)
- [CAMS: An Annotated Corpus for Causal Analysis of Mental Health Issues in Social Media Posts](https://mlsanity.com/paper/2207.04674/) (2022)
- [Suicidal Ideation Detection: A Review of Machine Learning Methods and Applications](https://mlsanity.com/paper/1910.12611/) (2019)
- [Improving the Generalizability of Depression Detection by Leveraging Clinical Questionnaires](https://mlsanity.com/paper/2204.10432/) (2022)
- [Two Directions for Clinical Data Generation with Large Language Models: Data-to-Label and Label-to-Data](https://mlsanity.com/paper/2401.06774/) (2023)
- [Domain-specific Continued Pretraining of Language Models for Capturing Long Context in Mental Health](https://mlsanity.com/paper/2304.10447/) (2023)
- [Hierarchical Attention Network for Explainable Depression Detection on Twitter Aided by Metaphor Concept Mappings](https://mlsanity.com/paper/2209.07494/) (2022)
- [ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?](https://mlsanity.com/paper/2311.16989/) (2023)
- [A Short Review of Ethical Challenges in Clinical Natural Language Processing](https://mlsanity.com/paper/1703.10090/) (2017)
- [Towards Preemptive Detection of Depression and Anxiety in Twitter](https://mlsanity.com/paper/2011.05249/) (2020)
- [Emotion fusion for mental illness detection from social media: A survey](https://mlsanity.com/paper/2304.09493/) (2023)

## Built on by (most cited first, 12 of 12)

- [A Survey of Large Language Models in Psychotherapy: Current Landscape and Future Directions](https://mlsanity.com/paper/2502.11095/) (2025)
- [MHQA: A Diverse, Knowledge Intensive Mental Health Question Answering Challenge for Language Models](https://mlsanity.com/paper/2502.15418/) (2025)
- [ProMind-LLM: Proactive Mental Health Care via Causal Reasoning with Sensor Data](https://mlsanity.com/paper/2505.14038/) (2025)
- [FedMentor: Domain-Aware Differential Privacy for Heterogeneous Federated LLMs in Mental Health](https://mlsanity.com/paper/2509.14275/) (2025)
- [A Multi-Level Benchmark for Causal Language Understanding in Social Media Discourse](https://mlsanity.com/paper/2509.16722/) (2025)
- [ReDepress: A Cognitive Framework for Detecting Depression Relapse from Social Media](https://mlsanity.com/paper/2509.17991/) (2025)
- [It Hears, It Sees too: Multi-Modal LLM for Depression Detection By Integrating Visual Understanding into Audio Language Models](https://mlsanity.com/paper/2511.19877/) (2025)
- [Responsible Evaluation of AI for Mental Health](https://mlsanity.com/paper/2602.00065/) (2026)
- [OMIND: Framework for Knowledge Grounded Finetuning and Multi-Turn Dialogue Benchmark for Mental Health LLMs](https://mlsanity.com/paper/2603.25105/) (2026)
- [When PCOS Meets Eating Disorders: An Explainable AI Approach to Detecting the Hidden Triple Burden](https://mlsanity.com/paper/2604.14356/) (2026)
- [The MADRS Pipeline: Supporting Depression Assessment in Clinical Trials](https://mlsanity.com/paper/2607.28190/) (2026)
- [From Pattern Recognizers to Personalized Companions: A Survey of Large Language Models in Mental Health](https://mlsanity.com/paper/2609.25186/) (2026)

## Abstract (arXiv)

In this study, we introduce ANGST, a novel, first-of-its kind benchmark for depression-anxiety comorbidity classification from social media posts. Unlike contemporary datasets that often oversimplify the intricate interplay between different mental health disorders by treating them as isolated conditions, ANGST enables multi-label classification, allowing each post to be simultaneously identified as indicating depression and/or anxiety. Comprising 2876 meticulously annotated posts by expert psychologists and an additional 7667 silver-labeled posts, ANGST posits a more representative sample of online mental health discourse. Moreover, we benchmark ANGST using various state-of-the-art language models, ranging from Mental-BERT to GPT-4. Our results provide significant insights into the capabilities and limitations of these models in complex diagnostic scenarios. While GPT-4 generally outperforms other models, none achieve an F1 score exceeding 72% in multi-class comorbid classification, underscoring the ongoing challenges in applying language models to mental health diagnostics.