# Linguistic Comparison of AI- and Human-Written Responses to Online Mental Health Queries arXiv: https://arxiv.org/abs/2504.09271 ยท HTML: https://mlsanity.com/paper/2504.09271/ ## Key facts - Cited by 6 papers in the map (data through 14 Oct 2026). - Builds on 24 of its 107 references that are in the map. - First posted on arXiv 12 Apr 2025. ## Builds on (most cited first, 24 of 24) - [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013) - [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021) - [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023) - [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018) - [A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support](https://mlsanity.com/paper/2009.08441/) (2020) - [Cultural Bias and Cultural Alignment of Large Language Models](https://mlsanity.com/paper/2311.14096/) (2023) - [Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries](https://mlsanity.com/paper/2310.13132/) (2023) - [Human-AI Collaboration Enables More Empathic Conversations in Text-based Peer-to-Peer Mental Health Support](https://mlsanity.com/paper/2203.15144/) (2022) - [The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships](https://mlsanity.com/paper/2410.20130/) (2024) - [Facilitating Self-Guided Mental Health Interventions Through Human-Language Model Interaction: A Case Study of Cognitive Restructuring](https://mlsanity.com/paper/2310.15461/) (2023) - [Modeling Empathy and Distress in Reaction to News Stories](https://mlsanity.com/paper/1808.10399/) (2018) - [Benefits and Harms of Large Language Models in Digital Mental Health](https://mlsanity.com/paper/2311.14693/) (2023) - [Improving Human-AI Collaboration With Descriptions of AI Behavior](https://mlsanity.com/paper/2301.06937/) (2023) - [Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI](https://mlsanity.com/paper/2302.00799/) (2023) - [Don't lose the message while paraphrasing: A study on content preserving style transfer](https://mlsanity.com/paper/2308.09055/) (2023) - [The Typing Cure: Experiences with Large Language Model Chatbots for Mental Health Support](https://mlsanity.com/paper/2401.14362/) (2024) - [TyDiP: A Dataset for Politeness Classification in Nine Typologically Diverse Languages](https://mlsanity.com/paper/2211.16496/) (2022) - [Lived Experience Not Found: LLMs Struggle to Align with Experts on Addressing Adverse Drug Reactions from Psychiatric Medication Use](https://mlsanity.com/paper/2410.19155/) (2024) - [Antisocial Behavior in Online Discussion Communities](https://mlsanity.com/paper/1504.00680/) (2015) - [Multi-Level Feedback Generation with Large Language Models for Empowering Novice Peer Counselors](https://mlsanity.com/paper/2403.15482/) (2024) - [Mental Health Coping Stories on Social Media: A Causal-Inference Study of Papageno Effect](https://mlsanity.com/paper/2302.09885/) (2023) - [AI on My Shoulder: Supporting Emotional Labor in Front-Office Roles with an LLM-based Empathetic Coworker](https://mlsanity.com/paper/2411.02408/) (2024) - [Is ChatGPT More Empathetic than Humans?](https://mlsanity.com/paper/2403.05572/) (2024) - [Empathy and the Right to Be an Exception: What LLMs Can and Cannot Do](https://mlsanity.com/paper/2401.14523/) (2024) ## Built on by (most cited first, 6 of 6) - [RubRIX: Rubric-Driven Risk Mitigation in Caregiver-AI Interactions](https://mlsanity.com/paper/2601.13235/) (2026) - [Interpersonal Theory of Suicide as a Lens to Examine Suicidal Ideation in Online Spaces](https://mlsanity.com/paper/2504.13277/) (2025) - [AI Psychosis: Does Conversational AI Amplify Delusion-Related Language?](https://mlsanity.com/paper/2603.19574/) (2026) - [How People Evaluate AI-, Expert-, and Peer-Style Financial Advice](https://mlsanity.com/paper/2608.09019/) (2026) - [PeerPen: AI-Assisted Writing for Online Mental Health Peer Support](https://mlsanity.com/paper/2609.14886/) (2026) - [Trustworthy FinAInce: Unpacking How AI-Mediated Financial Advice is Judged](https://mlsanity.com/paper/2609.20989/) (2026) ## Abstract (arXiv) The ubiquity and widespread use of digital and online technologies have transformed mental health support, with online mental health communities (OMHCs) providing safe spaces for peer support. More recently, generative AI and large language models (LLMs) have introduced new possibilities for scalable, around-the-clock mental health assistance that could potentially augment and supplement the capabilities of OMHCs. Although genAI shows promise in delivering immediate and personalized responses, its effectiveness in replicating the nuanced, experience-based support of human peers remains an open question. In this study, we harnessed 24,114 posts and 138,758 online community (OC) responses from 55 OMHCs on Reddit. We prompted several state-of-the-art LLMs (GPT-4-Turbo, Llama-3, and Mistral-7B) with these posts, and compared their responses to human-written (OC) responses based on a variety of linguistic measures across psycholinguistics and lexico-semantics. Our findings revealed that AI responses are more verbose, readable, and analytically structured, but lack linguistic diversity and personal narratives inherent in human--human interactions. Through a qualitative examination, we found validation as well as complementary insights into the nature of AI responses, such as its neutral stance and the absence of seeking back-and-forth clarifications. We discuss the ethical and practical implications of integrating generative AI into OMHCs, advocating for frameworks that balance AI's scalability and timeliness with the irreplaceable authenticity, social interactiveness, and expertise of human connections that form the ethos of online support communities.