# Deep Learning Mental Health Dialogue System

arXiv: https://arxiv.org/abs/2301.09412 · HTML: https://mlsanity.com/paper/2301.09412/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 30 references that are in the map.
- First posted on arXiv 23 Jan 2023.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [The Pushshift Reddit Dataset](https://mlsanity.com/paper/2001.08435/) (2020)
- [ParlAI: A Dialog Research Software Platform](https://mlsanity.com/paper/1705.06476/) (2017)
- [Controlled Hallucinations: Learning to Generate Faithfully from Noisy Data](https://mlsanity.com/paper/2010.05873/) (2020)

## Built on by (most cited first, 9 of 9)

- [Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers](https://mlsanity.com/paper/2504.18412/) (2025)
- [The opportunities and risks of large language models in mental health](https://mlsanity.com/paper/2403.14814/) (2024)
- [AnnaAgent: Dynamic Evolution Agent System with Multi-Session Memory for Realistic Seeker Simulation](https://mlsanity.com/paper/2506.00551/) (2025)
- [Challenges of Large Language Models for Mental Health Counseling](https://mlsanity.com/paper/2311.13857/) (2023)
- [Position: Beyond Assistance -- Reimagining LLMs as Ethical and Adaptive Co-Creators in Mental Health Care](https://mlsanity.com/paper/2503.16456/) (2025)
- [CALM-IT: Enhancing Synthetic Motivational Interviewing Dialogue Quality with Conversational Dynamics](https://mlsanity.com/paper/2601.10085/) (2026)
- [Towards Explainable and Safe Conversational Agents for Mental Health: A Survey](https://mlsanity.com/paper/2304.13191/) (2023)
- [Seeking Late Night Life Lines: Experiences of Conversational AI Use in Mental Health Crisis](https://mlsanity.com/paper/2512.23859/) (2025)
- [Evaluating the Effectiveness of the Foundational Models for Q&A Classification in Mental Health care](https://mlsanity.com/paper/2406.15966/) (2024)

## Abstract (arXiv)

Mental health counseling remains a major challenge in modern society due to cost, stigma, fear, and unavailability. We posit that generative artificial intelligence (AI) models designed for mental health counseling could help improve outcomes by lowering barriers to access. To this end, we have developed a deep learning (DL) dialogue system called Serena. The system consists of a core generative model and post-processing algorithms. The core generative model is a 2.7 billion parameter Seq2Seq Transformer fine-tuned on thousands of transcripts of person-centered-therapy (PCT) sessions. The series of post-processing algorithms detects contradictions, improves coherency, and removes repetitive answers. Serena is implemented and deployed on urlhttps://serena.chat, which currently offers limited free services. While the dialogue system is capable of responding in a qualitatively empathetic and engaging manner, occasionally it displays hallucination and long-term incoherence. Overall, we demonstrate that a deep learning mental health dialogue system has the potential to provide a low-cost and effective complement to traditional human counselors with less barriers to access.