# MindSET: Advancing Mental Health Benchmarking through Large-Scale Social Media Data

arXiv: https://arxiv.org/abs/2511.20672 · HTML: https://mlsanity.com/paper/2511.20672/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 43 references that are in the map.
- First posted on arXiv 14 Nov 2025.

## Builds on (most cited first, 4 of 4)

- [Depression and Self-Harm Risk Assessment in Online Forums](https://mlsanity.com/paper/1709.01848/) (2017)
- [SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions](https://mlsanity.com/paper/1806.05258/) (2018)
- [Handling and Presenting Harmful Text in NLP Research](https://mlsanity.com/paper/2204.14256/) (2022)
- [CARMA: Comprehensive Automatically-annotated Reddit Mental Health Dataset for Arabic](https://mlsanity.com/paper/2511.03102/) (2025)

## Built on by (most cited first, 2 of 2)

- [MentalSeek-Dx: Towards Progressive Hypothetico-Deductive Reasoning for Real-world Psychiatric Diagnosis](https://mlsanity.com/paper/2602.03340/) (2026)
- [Mirroring Minds: Asymmetric Linguistic Accommodation and Diagnostic Identity in ADHD and Autism Reddit Communities](https://mlsanity.com/paper/2604.10063/) (2026)

## Abstract (arXiv)

Social media data has become a vital resource for studying mental health, offering real-time insights into thoughts, emotions, and behaviors that traditional methods often miss. Progress in this area has been facilitated by benchmark datasets for mental health analysis; however, most existing benchmarks have become outdated due to limited data availability, inadequate cleaning, and the inherently diverse nature of social media content (e.g., multilingual and harmful material). We present a new benchmark dataset, MindSET, curated from Reddit using self-reported diagnoses to address these limitations. The annotated dataset contains over 13M annotated posts across seven mental health conditions, more than twice the size of previous benchmarks. To ensure data quality, we applied rigorous preprocessing steps, including language filtering, and removal of Not Safe for Work (NSFW) and duplicate content. We further performed a linguistic analysis using LIWC to examine psychological term frequencies across the eight groups represented in the dataset. To demonstrate the dataset utility, we conducted binary classification experiments for diagnosis detection using both fine-tuned language models and Bag-of-Words (BoW) features. Models trained on MindSET consistently outperformed those trained on previous benchmarks, achieving up to an 18-point improvement in F1 for Autism detection. Overall, MindSET provides a robust foundation for researchers exploring the intersection of social media and mental health, supporting both early risk detection and deeper analysis of emerging psychological trends.