Position: Beyond Assistance -- Reimagining LLMs as Ethical and Adaptive Co-Creators in Mental Health Care
Not on the map. Why?
Abstract (arXiv)
This position paper argues for a fundamental shift in how Large Language Models (LLMs) are integrated into the mental health care domain. We advocate for their role as co-creators rather than mere assistive tools. While LLMs have the potential to enhance accessibility, personalization, and crisis intervention, their adoption remains limited due to concerns about bias, evaluation, over-reliance, dehumanization, and regulatory uncertainties. To address these challenges, we propose two structured pathways: SAFE-i (Supportive, Adaptive, Fair, and Ethical Implementation) Guidelines for ethical and responsible deployment, and HAAS-e (Human-AI Alignment and Safety Evaluation) Framework for multidimensional, human-centered assessment. SAFE-i provides a blueprint for data governance, adaptive model engineering, and real-world integration, ensuring LLMs align with clinical and ethical standards. HAAS-e introduces evaluation metrics that go beyond technical accuracy to measure trustworthiness, empathy, cultural sensitivity, and actionability. We call for the adoption of these structured approaches to establish a responsible and scalable model for LLM-driven mental health support, ensuring that AI complements, rather than replaces, human expertise.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 37 of its 118 references that are in the map. First posted on arXiv 21 Feb 2025.
Builds on
37 of 118 references.
References · 37
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
- 2023A Survey of Large Language Modelscited by 2,576
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
- 2024A Survey on LLM-as-a-Judgecited by 1,423
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
- 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
- 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 288
- 2020A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Supportcited by 182
25 more
- 2023A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasetscited by 123
- 2023ChatGPT is not all you need. A State of the Art Review of large Generative AI modelscited by 86
- 2023Fairness And Bias in Artificial Intelligence: A Brief Survey of Sources, Impacts, And Mitigation Strategiescited by 85
- 2024A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendationscited by 85
- 2022Human-AI Collaboration Enables More Empathic Conversations in Text-based Peer-to-Peer Mental Health Supportcited by 68
- 2023MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Modelscited by 63
- 2024Against The Achilles' Heel: A Survey on Red Teaming for Generative Modelscited by 59
- 2024The opportunities and risks of large language models in mental healthcited by 54
- 2023Facilitating Self-Guided Mental Health Interventions Through Human-Language Model Interaction: A Case Study of Cognitive Restructuringcited by 53
- 2023MindfulDiary: Harnessing Large Language Model to Support Psychiatric Patients' Journalingcited by 50
- 2024A Toolbox for Surfacing Health Equity Harms and Biases in Large Language Modelscited by 45
- 2023Benefits and Harms of Large Language Models in Digital Mental Healthcited by 40
- 2016A statistical framework for fair predictive algorithmscited by 38
- 2023A Comprehensive Evaluation of Large Language Models on Benchmark Biomedical Text Processing Taskscited by 36
- 2021Discovering and Validating AI Errors With Crowdsourced Failure Reportscited by 28
- 2024Evaluating Human-AI Collaboration: A Review and Methodological Frameworkcited by 27
- 2024Position: Measure Dataset Diversity, Don't Just Claim Itcited by 26
- 2024Towards Safety and Helpfulness Balanced Responses via Controllable Large Language Modelscited by 14
- 2023Deep Learning Mental Health Dialogue Systemcited by 9
- 2024SouLLMate: An Adaptive LLM-Driven System for Advanced Mental Health Support and Assessment, Based on a Systematic Application Surveycited by 8
- 2024Applying and Evaluating Large Language Models in Mental Health Care: A Scoping Review of Human-Assessed Generative Taskscited by 6
- 2024Exploring Bias and Prediction Metrics to Characterise the Fairness of Machine Learning for Equity-Centered Public Health Decision-Making: A Narrative Reviewcited by 6
- 2024Structured Dialogue System for Mental Health: An LLM Chatbot Leveraging the PM+ Guidelinescited by 6
- 2021Coral: An Approach for Conversational Agents in Mental Health Applicationscited by 2
- 2024Optimizing Psychological Counseling with Instruction-Tuned Large Language Modelscited by 2
Built on by
Citing papers per year: 2026: 3.
3 papers.