# Depression detection in social media posts using transformer-based models and auxiliary features

arXiv: https://arxiv.org/abs/2409.20048 · HTML: https://mlsanity.com/paper/2409.20048/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 72 references that are in the map.
- First posted on arXiv 30 Sep 2024.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification](https://mlsanity.com/paper/2010.12421/) (2020)
- [MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare](https://mlsanity.com/paper/2110.15621/) (2021)
- [Depression and Self-Harm Risk Assessment in Online Forums](https://mlsanity.com/paper/1709.01848/) (2017)
- [Utilizing Neural Networks and Linguistic Metadata for Early Detection of Depression Indications in Text Sequences](https://mlsanity.com/paper/1804.07000/) (2018)
- [Explainable Identification of Dementia from Transcripts using Transformer Networks](https://mlsanity.com/paper/2109.06980/) (2021)
- [Understanding and Measuring Psychological Stress using Social Media](https://mlsanity.com/paper/1811.07430/) (2018)
- [Utilizing BERT Intermediate Layers for Aspect Based Sentiment Analysis and Natural Language Inference](https://mlsanity.com/paper/2002.04815/) (2020)
- [Detecting Dementia from Speech and Transcripts using Transformers](https://mlsanity.com/paper/2110.14769/) (2021)
- [DEPTWEET: A Typology for Social Media Texts to Detect Depression Severities](https://mlsanity.com/paper/2210.05372/) (2022)
- [Calibration of Transformer-based Models for Identifying Stress and Depression in Social Media](https://mlsanity.com/paper/2305.16797/) (2023)
- [Detection and Classification of mental illnesses on social media using RoBERTa](https://mlsanity.com/paper/2011.11226/) (2020)
- [A Multimodal Approach for Dementia Detection from Spontaneous Speech with Tensor Fusion Layer](https://mlsanity.com/paper/2211.04368/) (2022)
- [Mental Illness Classification on Social Media Texts using Deep Learning and Transfer Learning](https://mlsanity.com/paper/2207.01012/) (2022)
- [Detecting Reddit Users with Depression Using a Hybrid Neural Network SBERT-CNN](https://mlsanity.com/paper/2302.02759/) (2023)
- [Multitask learning for recognizing stress and depression in social media](https://mlsanity.com/paper/2305.18907/) (2023)
- [Social Bots: Detection and Challenges](https://mlsanity.com/paper/2312.17423/) (2023)
- [Explainable Verbal Deception Detection using Transformers](https://mlsanity.com/paper/2210.03080/) (2022)

## Built on by (most cited first, 2 of 2)

- [A Graph Neural Architecture Search Approach for Identifying Bots in Social Media](https://mlsanity.com/paper/2411.16285/) (2024)
- [Explainable Detection of Depression Status Shifts from User Digital Traces](https://mlsanity.com/paper/2605.14995/) (2026)

## Abstract (arXiv)

The detection of depression in social media posts is crucial due to the increasing prevalence of mental health issues. Traditional machine learning algorithms often fail to capture intricate textual patterns, limiting their effectiveness in identifying depression. Existing studies have explored various approaches to this problem but often fall short in terms of accuracy and robustness. To address these limitations, this research proposes a neural network architecture leveraging transformer-based models combined with metadata and linguistic markers. The study employs DistilBERT, extracting information from the last four layers of the transformer, applying learned weights, and averaging them to create a rich representation of the input text. This representation, augmented by metadata and linguistic markers, enhances the model's comprehension of each post. Dropout layers prevent overfitting, and a Multilayer Perceptron (MLP) is used for final classification. Data augmentation techniques, inspired by the Easy Data Augmentation (EDA) methods, are also employed to improve model performance. Using BERT, random insertion and substitution of phrases generate additional training data, focusing on balancing the dataset by augmenting underrepresented classes. The proposed model achieves weighted Precision, Recall, and F1-scores of 84.26%, 84.18%, and 84.15%, respectively. The augmentation techniques significantly enhance model performance, increasing the weighted F1-score from 72.59% to 84.15%.