# Detection of depression on social networks using transformers and ensembles

arXiv: https://arxiv.org/abs/2305.05325 · HTML: https://mlsanity.com/paper/2305.05325/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 97 references that are in the map.
- First posted on arXiv 9 May 2023.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare](https://mlsanity.com/paper/2110.15621/) (2021)
- [Corpus Creation for Sentiment Analysis in Code-Mixed Tamil-English Text](https://mlsanity.com/paper/2006.00206/) (2020)
- [An Empirical Evaluation of doc2vec with Practical Insights into Document Embedding Generation](https://mlsanity.com/paper/1607.05368/) (2016)
- [Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling](https://mlsanity.com/paper/1611.06639/) (2016)
- [Sentiment of Emojis](https://mlsanity.com/paper/1509.07761/) (2015)
- [Data set creation and empirical analysis for detecting signs of depression from social media postings](https://mlsanity.com/paper/2202.03047/) (2022)
- [Hate Speech and Offensive Language Detection in Bengali](https://mlsanity.com/paper/2210.03479/) (2022)
- [BB_twtr at SemEval-2017 Task 4: Twitter Sentiment Analysis with CNNs and LSTMs](https://mlsanity.com/paper/1704.06125/) (2017)
- [Hate Speech Detection in Roman Urdu](https://mlsanity.com/paper/2108.02830/) (2021)
- [Unveiling the Political Agenda of the European Parliament Plenary: A Topical Analysis](https://mlsanity.com/paper/1505.07302/) (2015)
- [Bangla Text Dataset and Exploratory Analysis for Online Harassment Detection](https://mlsanity.com/paper/2102.02478/) (2021)
- [Enhanced Twitter Sentiment Classification Using Contextual Information](https://mlsanity.com/paper/1605.05195/) (2016)
- [Training Cross-Lingual embeddings for Setswana and Sepedi](https://mlsanity.com/paper/2111.06230/) (2021)

## Abstract (arXiv)

As the impact of technology on our lives is increasing, we witness increased use of social media that became an essential tool not only for communication but also for sharing information with community about our thoughts and feelings. This can be observed also for people with mental health disorders such as depression where they use social media for expressing their thoughts and asking for help. This opens a possibility to automatically process social media posts and detect signs of depression. We build several large pre-trained language model based classifiers for depression detection from social media posts. Besides fine-tuning BERT, RoBERTA, BERTweet, and mentalBERT were also construct two types of ensembles. We analyze the performance of our models on two data sets of posts from social platforms Reddit and Twitter, and investigate also the performance of transfer learning across the two data sets. The results show that transformer ensembles improve over the single transformer-based classifiers.