# Detecting potentially harmful and protective suicide-related content on twitter: A machine learning approach

arXiv: https://arxiv.org/abs/2112.04796 · HTML: https://mlsanity.com/paper/2112.04796/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 51 references that are in the map.
- First posted on arXiv 9 Dec 2021.

## Builds on (most cited first, 6 of 6)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification](https://mlsanity.com/paper/2010.12421/) (2020)
- [Suicidal Ideation Detection: A Review of Machine Learning Methods and Applications](https://mlsanity.com/paper/1910.12611/) (2019)
- [ktrain: A Low-Code Library for Augmented Machine Learning](https://mlsanity.com/paper/2004.10703/) (2020)

## Built on by (most cited first, 5 of 5)

- [Mental Health Coping Stories on Social Media: A Causal-Inference Study of Papageno Effect](https://mlsanity.com/paper/2302.09885/) (2023)
- [Survey and Experiments on Mental Disorder Detection via Social Media: From Large Language Models and RAG to Agents](https://mlsanity.com/paper/2504.02800/) (2025)
- [From Reddit to Generative AI: Evaluating Large Language Models for Anxiety Support Fine-tuned on Social Media Data](https://mlsanity.com/paper/2505.18464/) (2025)
- [LEXpander: applying colexification networks to automated lexicon expansion](https://mlsanity.com/paper/2205.15850/) (2022)
- [Leveraging Large Language Models for Spontaneous Speech-Based Suicide Risk Detection](https://mlsanity.com/paper/2507.00693/) (2025)

## Abstract (arXiv)

Research shows that exposure to suicide-related news media content is associated with suicide rates, with some content characteristics likely having harmful and others potentially protective effects. Although good evidence exists for a few selected characteristics, systematic large scale investigations are missing in general, and in particular for social media data. We apply machine learning methods to classify large quantities of Twitter data according to a novel annotation scheme that distinguishes 12 categories of suicide-related tweets. We then trained a benchmark of machine learning models including a majority classifier, an approach based on word frequency (TF-IDF with a linear SVM) and two state-of-the-art deep learning models (BERT, XLNet). The two deep learning models achieved the best performance in two classification tasks: In the first task, we classified six main content categories, including personal stories about either suicidal ideation and attempts or coping, calls for action intending to spread either problem awareness or prevention-related information, reporting of suicide cases, and other tweets irrelevant to these categories. The deep learning models reached accuracy scores above 73% on average across the six categories, and F1-scores in between 0.70 and 0.85 for all but the suicidal ideation and attempts category (0.51-0.55). In the second task, separating tweets referring to actual suicide from off-topic tweets, they correctly labeled around 88% of tweets, with BERT achieving F1-scores of 0.93 and 0.74 for the two categories, respectively. These classification performances are comparable to the state-of-the-art on similar tasks. By making data labeling more efficient, this work has enabled large-scale investigations on harmful and protective associations of social media content with suicide rates and help-seeking behavior.