mlsanityData to 7 Oct 2026

Towards Algorithmic Fidelity: Mental Health Representation across Demographics in Synthetic vs. Human-generated Data

arXiv 2403.16909 ↗First posted 25 Mar 2024cs.AI cs.CL cs.CYcited by 4

Not on the map. Why?

Abstract (arXiv)

Synthetic data generation has the potential to impact applications and domains with scarce data. However, before such data is used for sensitive tasks such as mental health, we need an understanding of how different demographics are represented in it. In our paper, we analyze the potential of producing synthetic data using GPT-3 by exploring the various stressors it attributes to different race and gender combinations, to provide insight for future researchers looking into using LLMs for data generation. Using GPT-3, we develop HEADROOM, a synthetic dataset of 3,120 posts about depression-triggering stressors, by controlling for race, gender, and time frame (before and after COVID-19). Using this dataset, we conduct semantic and lexical analyses to (1) identify the predominant stressors for each demographic group; and (2) compare our synthetic data to a human-generated dataset. We present the procedures to generate queries to develop depression data using GPT-3, and conduct analyzes to uncover the types of stressors it assigns to demographic groups, which could be used to test the limitations of LLMs for synthetic data generation for depression data. Our findings show that synthetic data mimics some of the human-generated data distribution for the predominant depression stressors across diverse demographics.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 9 of its 33 references that are in the map. First posted on arXiv 25 Mar 2024.

Builds on

9 of 33 references.

References · 9

  1. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  2. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
  3. 2022Large Language Models are Few-Shot Clinical Information Extractorscited by 206
  4. 2023Does Synthetic Data Generation of LLMs Help Clinical Text Mining?cited by 151
  5. 2021A Survey of Race, Racism, and Anti-Racism in NLPcited by 74
  6. 2022Differentially Private Language Models for Secure Data Sharingcited by 68
  7. 2021Gender and Racial Fairness in Depression Research using Social Mediacited by 17
  8. 2022Gendered Mental Health Stigma in Masked Language Modelscited by 17
  9. 2020Using Noisy Self-Reports to Predict Twitter User Demographicscited by 8

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 3.

4 papers.

  1. 2025Thousand Voices of Trauma: A Large-Scale Synthetic Dataset for Modeling Prolonged Exposure Therapy Conversationscited by 11
  2. 2025Roleplaying with Structure: Synthetic Therapist-Client Conversation Generation from Questionnairescited by 4
  3. 2025Evaluating Large Language Models for Anxiety, Depression, and Stress Detection: Insights into Prompting Strategies and Synthetic Datacited by 2
  4. 2024Chatting Up Attachment: Using LLMs to Predict Adult Bondscited by 0