mlsanityData to 7 Oct 2026

Responsible Evaluation of AI for Mental Health

arXiv 2602.00065 ↗First posted 20 Jan 2026cs.CY cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Although artificial intelligence (AI) shows growing promise for mental health care, current approaches to evaluating AI tools in this domain remain fragmented and poorly aligned with clinical practice, social context, and first-hand user experience. This paper argues for a rethinking of responsible evaluation -- what is measured, by whom, and for what purpose -- by introducing an interdisciplinary framework that integrates clinical soundness, social context, and equity, providing a structured basis for evaluation. Through an analysis of 135 recent *CL publications, we identify recurring limitations, including over-reliance on generic metrics that do not capture clinical validity, therapeutic appropriateness, or user experience, limited participation from mental health professionals, and insufficient attention to safety and equity. To address these gaps, we propose a taxonomy of AI mental health support types -- assessment-, intervention-, and information synthesis-oriented -- each with distinct risks and evaluative requirements, and illustrate its use through case studies.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 87 of its 180 references that are in the map. First posted on arXiv 20 Jan 2026.

Builds on

87 of 180 references.

References · 87

  1. 2016Inherent Trade-Offs in the Fair Determination of Risk Scorescited by 582
  2. 2021Towards Emotional Support Dialog Systemscited by 294
  3. 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
  4. 2020A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Supportcited by 182
  5. 2023Cognitive Reframing of Negative Thoughts through Human-Language Model Interactioncited by 77
  6. 2024Roleplay-doh: Enabling Domain-Experts to Create LLM-simulated Patients via Eliciting and Adhering to Principlescited by 72
  7. 2024Cactus: Towards Psychological Counseling Conversations using Cognitive Behavioral Theorycited by 60
  8. 2023Facilitating Self-Guided Mental Health Interventions Through Human-Language Model Interaction: A Case Study of Cognitive Restructuringcited by 53
  9. 2024HealMe: Harnessing Cognitive Reframing in Large Language Models for Psychotherapycited by 47
  10. 2023Knowledge-enhanced Mixed-initiative Dialogue System for Emotional Support Conversationscited by 41
  11. 2025Culture is Not Trivia: Sociocultural Theory for Cultural NLPcited by 40
  12. 2022PAL: Persona-Augmented Emotional Support Conversation Generationcited by 39
38 more
  1. 2022D4: a Chinese Dialogue Dataset for Depression-Diagnosis-Oriented Chatcited by 37
  2. 2024PsyDT: Using LLMs to Construct the Digital Twin of Psychological Counselor with Personalized Counseling Style for Psychological Counselingcited by 37
  3. 2023Empowering Psychotherapy with Large Language Models: Cognitive Distortion Detection through Diagnosis of Thought Promptingcited by 31
  4. 2023Ask an Expert: Leveraging Language Models to Improve Strategic Reasoning in Goal-Oriented Dialogue Modelscited by 30
  5. 2024PsychoGAT: A Novel Psychological Measurement Paradigm through Interactive Fiction Games with LLM Agentscited by 28
  6. 2022Improving the Generalizability of Depression Detection by Leveraging Clinical Questionnairescited by 26
  7. 2023Task-Adaptive Tokenization: Enhancing Long-Form Text Generation Efficacy in Mental Health and Beyondcited by 22
  8. 2025EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safetycited by 21
  9. 2025AnnaAgent: Dynamic Evolution Agent System with Multi-Session Memory for Realistic Seeker Simulationcited by 21
  10. 2022Symptom Identification for Interpretable Detection of Multiple Mental Disorderscited by 20
  11. 2024The Colorful Future of LLMs: Evaluating and Improving LLMs as Emotional Supporters for Queer Youthcited by 20
  12. 2024Understanding the Therapeutic Relationship between Counselors and Clients in Online Text-based Counseling using LLMscited by 19
  13. 2024Do Large Language Models Align with Core Mental Health Counseling Competencies?cited by 19
  14. 2021Gender and Racial Fairness in Depression Research using Social Mediacited by 17
  15. 2022Identifying Moments of Change from Longitudinal User Textcited by 17
  16. 2022Gendered Mental Health Stigma in Masked Language Modelscited by 17
  17. 2024Lived Experience Not Found: LLMs Struggle to Align with Experts on Addressing Adverse Drug Reactions from Psychiatric Medication Usecited by 17
  18. 2023An Annotated Dataset for Explainable Interpersonal Risk Factors of Mental Disturbance in Social Media Postscited by 16
  19. 2024When LLMs Meets Acoustic Landmarks: An Efficient Approach to Integrate Speech into Large Language Models for Depression Detectioncited by 16
  20. 2024Multi-Level Feedback Generation with Large Language Models for Empowering Novice Peer Counselorscited by 16
  21. 2021Empirical Evaluation of Pre-trained Transformers for Human-Level NLP: The Role of Sample Size and Dimensionalitycited by 15
  22. 2025Consistent Client Simulation for Motivational Interviewing-based Counselingcited by 14
  23. 2025MAGI: Multi-Agent Guided Interview for Psychiatric Assessmentcited by 14
  24. 2021Micromodels for Efficient, Explainable, and Reusable Systems: A Case Study on Mental Healthcited by 13
  25. 2025Can Large Language Models Identify Implicit Suicidal Ideation? An Empirical Evaluationcited by 13
  26. 2025MIRROR: Multimodal Cognitive Reframing Therapy for Rolling with Resistancecited by 13
  27. 2024IMBUE: Improving Interpersonal Effectiveness through Simulation and Just-in-time Feedback with Human-Language Model Interactioncited by 12
  28. 2024LLM Questionnaire Completion for Automatic Psychiatric Assessmentcited by 12
  29. 2024Diverse Perspectives, Divergent Models: Cross-Cultural Evaluation of Depression Detection on Twittercited by 12
  30. 2024Still Not Quite There! Evaluating Large Language Models for Comorbid Mental Health Diagnosiscited by 12
  31. 2023Language and Mental Health: Measures of Emotion Dynamics from Text as Linguistic Biosocial Markerscited by 11
  32. 2024Chinese MentalBERT: Domain-Adaptive Pre-training on Social Media for Chinese Mental Health Text Analysiscited by 11
  33. 2024Knowledge Planning in Large Language Models for Domain-Aligned Counseling Summarizationcited by 11
  34. 2025CAMI: A Counselor Agent Supporting Motivational Interviewing through State Inference and Topic Explorationcited by 11
  35. 2025Crisp: Cognitive Restructuring of Negative Thoughts through Multi-turn Supportive Dialoguescited by 11
  36. 2025Feel the Difference? A Comparative Analysis of Emotional Arcs in Real and LLM-Generated CBT Sessionscited by 11
  37. 2023A Cognitive Stimulation Dialogue System with Multi-source Knowledge Fusion for Elders with Cognitive Impairmentcited by 10
  38. 2023FedTherapist: Mental Health Monitoring with User-Generated Linguistic Expressions on Smartphones via Federated Learningcited by 10

50 most cited of 87 All 87 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026PeerPen: AI-Assisted Writing for Online Mental Health Peer Supportcited by 0
  2. 2026From Pattern Recognizers to Personalized Companions: A Survey of Large Language Models in Mental Healthcited by 0