mlsanityData to 14 Oct 2026

MentalChat16K: A Benchmark Dataset for Conversational Mental Health Assistance

arXiv 2503.13509 ↗First posted 13 Mar 2025cs.LG cs.AI cs.CLcited by 39

Not on the map. Why?

Abstract (arXiv)

We introduce MentalChat16K, an English benchmark dataset combining a synthetic mental health counseling dataset and a dataset of anonymized transcripts from interventions between Behavioral Health Coaches and Caregivers of patients in palliative or hospice care. Covering a diverse range of conditions like depression, anxiety, and grief, this curated dataset is designed to facilitate the development and evaluation of large language models for conversational mental health assistance. By providing a high-quality resource tailored to this critical domain, MentalChat16K aims to advance research on empathetic, personalized AI solutions to improve access to mental health support services. The dataset prioritizes patient privacy, ethical considerations, and responsible data usage. MentalChat16K presents a valuable opportunity for the research community to innovate AI technologies that can positively impact mental well-being. The dataset is available at https://huggingface.co/datasets/ShenLab/MentalChat16K and the code and documentation are hosted on GitHub at https://github.com/ChiaPatricia/MentalChat16K.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Builds on 25 of its 76 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

25 of 76 references.

References · 25

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  9. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  10. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  11. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  12. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
13 more
  1. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 575
  2. 2023Zephyr: Direct Distillation of LM Alignmentcited by 506
  3. 2021MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcarecited by 161
  4. 2021PsyQA: A Chinese Dataset for Generating Long Counseling Text for Mental Health Supportcited by 92
  5. 2022Tomayto, Tomahto. Beyond Token-level Answer Equivalence for Question Answering Evaluationcited by 67
  6. 2021Semantic Answer Similarity for Evaluating Question Answering Modelscited by 47
  7. 2024CBT-LLM: A Chinese Large Language Model for Cognitive Behavioral Therapy-based Mental Health Question Answeringcited by 45
  8. 2024Large Language Models Produce Responses Perceived to be Empathiccited by 36
  9. 2021Speaker and Time-aware Joint Contextual Learning for Dialogue-act Classification in Counselling Conversationscited by 31
  10. 2023Enhancing Psychological Counseling with Large Language Model: A Multifaceted Decision-Support System for Non-Professionalscited by 24
  11. 2022Counseling Summarization using Mental Health Knowledge Guided Utterance Filteringcited by 18
  12. 2023Harnessing Large Language Models' Empathetic Response Generation Capabilities for Online Mental Health Counselling Supportcited by 17
  13. 2021Clinical Evidence Engine: Proof-of-Concept For A Clinical-Domain-Agnostic Decision Support Infrastructurecited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 13; 2026: 26.

39 papers.

Sort:

By area:

  1. 2025When Can We Trust LLMs in Mental Health? Large-Scale Benchmarks for Reliable LLM Evaluationcited by 19
  2. 2025MindEval: Benchmarking Language Models on Multi-turn Mental Health Supportcited by 10
  3. 2025MAGneT: Coordinated Multi-Agent Generation of Synthetic Multi-Turn Mental Health Counseling Sessionscited by 6
  4. 2025PsychiatryBench: A Multi-Task Benchmark for LLMs in Psychiatrycited by 6
  5. 2026Assessing the Quality of Mental Health Support in LLM Responses through Multi-Attribute Human Evaluationcited by 6
  6. 2025PersonaFuse: A Personality Activation-Driven Framework for Enhancing Human-LLM Interactionscited by 5
  7. 2025Domain-Specific Constitutional AI: Enhancing Safety in LLM-Powered Mental Health Chatbotscited by 4
  8. 2026LingxiDiagBench: A Multi-Agent Framework for Benchmarking LLMs in Chinese Psychiatric Consultation and Diagnosiscited by 4
  9. 2025A Comprehensive Review of Datasets for Clinical Mental Health AI Systemscited by 3
  10. 2026CALM-IT: Enhancing Synthetic Motivational Interviewing Dialogue Quality with Conversational Dynamicscited by 3
  11. 2026MPIB: A Benchmark for Medical Prompt Injection Attacks and Clinical Safety in LLMscited by 3
  12. 2026Trusting What You Cannot See: Auditable Fine-Tuning and Inference for Proprietary AIcited by 3
27 more
  1. 2026MHSafeEval: Role-Aware Interaction-Level Evaluation of Mental Health Safety in Large Language Modelscited by 3
  2. 2025T-FIX: Text-Based Explanations with Features Interpretable to eXpertscited by 2
  3. 2026Graph2Counsel: Clinically Grounded Synthetic Counseling Dialogue Generation from Client Psychological Graphscited by 2
  4. 2025RHealthTwin: Towards Responsible and Multimodal Digital Twins for Personalized Well-beingcited by 1
  5. 2025EthicsMH: A Pilot Benchmark for Ethical Reasoning in Mental Health AIcited by 1
  6. 2026EFT-CoT: A Multi-Agent Chain-of-Thought Framework for Emotion-Focused Therapycited by 1
  7. 2026OMIND: Framework for Knowledge Grounded Finetuning and Multi-Turn Dialogue Benchmark for Mental Health LLMscited by 1
  8. 2026Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMscited by 1
  9. 2026Does Theory of Mind Improvement Really Benefit Human-AI Interactions? Empirical Findings from Interactive Evaluationscited by 1
  10. 2026MentalHospital: A Virtual Environment for Evaluating Psychiatric Clinical Encounterscited by 1
  11. 2026Privacy-Preserving Split Learning for Federated LLM Fine-Tuningcited by 1
  12. 2025Mentalic Net: Development of RAG-based Conversational AI and Evaluation Framework for Mental Health Supportcited by 0
  13. 2025Plural Voices, Single Agent: Towards Inclusive AI in Multi-User Domestic Spacescited by 0
  14. 2025InvisibleBench: A Deployment Gate for Caregiving Relationship AIcited by 0
  15. 2026VERA-MH: Validation of Ethical and Responsible AI in Mental Healthcited by 0
  16. 2026EmoTrack: Clinical-Semantic Modeling for Text-Based Depression Severity Estimationcited by 0
  17. 2026Supervised Learning Objectives for High-Dimensional Treatment Effectscited by 0
  18. 2026Does Capability Transfer to Subjective Behavior -- and Would Our Instruments Tell Us? A Self-Evolving, Trust-by-Construction Evaluation Paradigmcited by 0
  19. 2026Expert-Level Crisis Detection in Mental Health Conversationscited by 0
  20. 2026Towards Understanding and Measuring COGNITIVE ATROPHY in LLM Behaviourcited by 0
  21. 2026When the Aggregator Cheats: Data-Free Backdoors in Federated LLM-based QA Systemscited by 0
  22. 2026Low-Agreeableness Persona Conditioning for Safe LLM Fine-Tuningcited by 0
  23. 2026Training Therapeutic Judges and Multi-Agent Systems for Human-Aligned Mental Health Supportcited by 0
  24. 2026Cognivia: A Cognitive Behavioral Therapy Copilot for Evidence-Based Mental Healthcarecited by 0
  25. 2026AnchorSIPS: A Synthetic Dataset and Evaluation Resource for Evidence-Supported Psychosis-Risk Symptom Measurementcited by 0
  26. 2026When Retrieval Helps: Selective Retrieval for Single-Turn Mental-Health QAcited by 0
  27. 2026From Pattern Recognizers to Personalized Companions: A Survey of Large Language Models in Mental Healthcited by 0