mlsanityData to 13 Oct 2026

ChatGPT for Suicide Risk Assessment on Social Media: Quantitative Evaluation of Model Performance, Potentials and Limitations

arXiv 2306.09390 ↗First posted 15 Jun 2023cs.CL cs.AI cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

This paper presents a novel framework for quantitatively evaluating the interactive ChatGPT model in the context of suicidality assessment from social media posts, utilizing the University of Maryland Reddit suicidality dataset. We conduct a technical evaluation of ChatGPT's performance on this task using Zero-Shot and Few-Shot experiments and compare its results with those of two fine-tuned transformer-based models. Additionally, we investigate the impact of different temperature parameters on ChatGPT's response generation and discuss the optimal temperature based on the inconclusiveness rate of ChatGPT. Our results indicate that while ChatGPT attains considerable accuracy in this task, transformer-based models fine-tuned on human-annotated datasets exhibit superior performance. Moreover, our analysis sheds light on how adjusting the ChatGPT's hyperparameters can improve its ability to assist mental health professionals in this critical task.

Key facts. Cited by 9 papers in the map (data through 13 Oct 2026). Builds on 14 of its 49 references that are in the map. First posted on arXiv 15 Jun 2023.

Builds on

14 of 49 references.

References · 14

  1. 2017Attention Is All You Needcited by 60,106
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  3. 2020Language Models are Few-Shot Learnerscited by 32,320
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,281
  5. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,779
  6. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,215
  7. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  8. 2023How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detectioncited by 478
  9. 2021MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcarecited by 159
  10. 2022ChatGPT Makes Medicine Easy to Swallow: An Exploratory Case Study on Simplified Radiology Reportscited by 101
  11. 2019Zero-shot Text Classification With Generative Language Modelscited by 55
  12. 2020Suicidal Ideation and Mental Disorder Detection with Attentive Relation Networkscited by 28
2 more
  1. 2022Suicidal Ideation Detection on Social Media: A Review of Machine Learning Methodscited by 4
  2. 2020Natural language processing for word sense disambiguation and information extractioncited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 1; 2026: 1.

9 papers.

  1. 2023User Modeling in the Era of Large Language Models: Current Research and Future Directionscited by 18
  2. 2024Large Language Models: A New Approach for Privacy Policy Analysis at Scalecited by 14
  3. 2024Socially Aware Synthetic Data Generation for Suicidal Ideation Detection Using Large Language Modelscited by 10
  4. 2024Chumor 2.0: Towards Benchmarking Chinese Humor Understandingcited by 4
  5. 2024Chumor 1.0: A Truly Funny and Challenging Chinese Humor Understanding Dataset from Ruo Zhi Bacited by 3
  6. 2025ProfiLLM: An LLM-Based Framework for Implicit Profiling of Chatbot Userscited by 3
  7. 2024Adapting Mental Health Prediction Tasks for Cross-lingual Learning via Meta-Training and In-context Learning with Large Language Modelcited by 1
  8. 2026Beyond Accuracy: Interpreting Topic Representation in Suicide Ideation Detection Modelscited by 1
  9. 2024A Survey on the Real Power of ChatGPTcited by 0