# Beyond Architectures: Evaluating the Role of Contextual Embeddings in Detecting Bipolar Disorder on Social Media

arXiv: https://arxiv.org/abs/2507.14231 · HTML: https://mlsanity.com/paper/2507.14231/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 24 references that are in the map.
- First posted on arXiv 17 Jul 2025.

## Builds on (most cited first, 4 of 4)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions](https://mlsanity.com/paper/1806.05258/) (2018)
- [A Comparative Analysis of Transformer and LSTM Models for Detecting Suicidal Ideation on Reddit](https://mlsanity.com/paper/2411.15404/) (2024)
- [A Survey-Based Qualitative Study to Characterize Expectations of Software Developers from Five Stakeholders](https://mlsanity.com/paper/2107.09587/) (2021)

## Built on by (most cited first, 1 of 1)

- [A Benchmark Suite of Reddit-Derived Datasets for Mental Health Detection](https://mlsanity.com/paper/2604.23458/) (2026)

## Abstract (arXiv)

Bipolar disorder is a chronic mental illness frequently underdiagnosed due to subtle early symptoms and social stigma. This paper explores the advanced natural language processing (NLP) models for recognizing signs of bipolar disorder based on user-generated social media text. We conduct a comprehensive evaluation of transformer-based models (BERT, RoBERTa, ALBERT, ELECTRA, DistilBERT) and Long Short Term Memory (LSTM) models based on contextualized (BERT) and static (GloVe, Word2Vec) word embeddings. Experiments were performed on a large, annotated dataset of Reddit posts after confirming their validity through sentiment variance and judgmental analysis. Our results demonstrate that RoBERTa achieves the highest performance among transformer models with an F1 score of ~98% while LSTM models using BERT embeddings yield nearly identical results. In contrast, LSTMs trained on static embeddings fail to capture meaningful patterns, scoring near-zero F1. These findings underscore the critical role of contextual language modeling in detecting bipolar disorder. In addition, we report model training times and highlight that DistilBERT offers an optimal balance between efficiency and accuracy. In general, our study offers actionable insights for model selection in mental health NLP applications and validates the potential of contextualized language models to support early bipolar disorder screening.