# Mental Multi-class Classification on Social Media: Benchmarking Transformer Architectures against LSTM Models

arXiv: https://arxiv.org/abs/2509.16542 · HTML: https://mlsanity.com/paper/2509.16542/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 26 references that are in the map.
- First posted on arXiv 20 Sep 2025.

## Builds on (most cited first, 7 of 7)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions](https://mlsanity.com/paper/1806.05258/) (2018)
- [Utilizing Neural Networks and Linguistic Metadata for Early Detection of Depression Indications in Text Sequences](https://mlsanity.com/paper/1804.07000/) (2018)
- [Suicidal Ideation and Mental Disorder Detection with Attentive Relation Networks](https://mlsanity.com/paper/2004.07601/) (2020)
- [A Survey-Based Qualitative Study to Characterize Expectations of Software Developers from Five Stakeholders](https://mlsanity.com/paper/2107.09587/) (2021)
- [Advancing Mental Disorder Detection: A Comparative Evaluation of Transformer and LSTM Architectures on Social Media](https://mlsanity.com/paper/2507.19511/) (2025)

## Built on by (most cited first, 1 of 1)

- [A Benchmark Suite of Reddit-Derived Datasets for Mental Health Detection](https://mlsanity.com/paper/2604.23458/) (2026)

## Abstract (arXiv)

Millions of people openly share mental health struggles on social media, providing rich data for early detection of conditions such as depression, bipolar disorder, etc. However, most prior Natural Language Processing (NLP) research has focused on single-disorder identification, leaving a gap in understanding the efficacy of advanced NLP techniques for distinguishing among multiple mental health conditions. In this work, we present a large-scale comparative study of state-of-the-art transformer versus Long Short-Term Memory (LSTM)-based models to classify mental health posts into exclusive categories of mental health conditions. We first curate a large dataset of Reddit posts spanning six mental health conditions and a control group, using rigorous filtering and statistical exploratory analysis to ensure annotation quality. We then evaluate five transformer architectures (BERT, RoBERTa, DistilBERT, ALBERT, and ELECTRA) against several LSTM variants (with or without attention, using contextual or static embeddings) under identical conditions. Experimental results show that transformer models consistently outperform the alternatives, with RoBERTa achieving 91-99% F1-scores and accuracies across all classes. Notably, attention-augmented LSTMs with BERT embeddings approach transformer performance (up to 97% F1-score) while training 2-3.5 times faster, whereas LSTMs using static embeddings fail to learn useful signals. These findings represent the first comprehensive benchmark for multi-class mental health detection, offering practical guidance on model selection and highlighting an accuracy-efficiency trade-off for real-world deployment of mental health NLP systems.