# Towards Automated Real-time Evaluation in Text-based Counseling

arXiv: https://arxiv.org/abs/2203.03442 · HTML: https://mlsanity.com/paper/2203.03442/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 34 references that are in the map.
- First posted on arXiv 7 Mar 2022.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Models](https://mlsanity.com/paper/1902.10547/) (2019)
- [Observing Dialogue in Therapy: Categorizing and Forecasting Behavioral Codes](https://mlsanity.com/paper/1907.00326/) (2019)
- [Balancing Objectives in Counseling Conversations: Advancing Forwards or Looking Backwards](https://mlsanity.com/paper/2005.04245/) (2020)
- [Automated Quality Assessment of Cognitive Behavioral Therapy Sessions Through Highly Contextualized Language Representations](https://mlsanity.com/paper/2102.11573/) (2021)
- [Multi-label Multi-task Deep Learning for Behavioral Coding](https://mlsanity.com/paper/1810.12349/) (2018)
- [Automated Evaluation Of Psychotherapy Skills Using Speech And Language Technologies](https://mlsanity.com/paper/2102.11265/) (2021)
- [Conversation Model Fine-Tuning for Classifying Client Utterances in Counseling Dialogues](https://mlsanity.com/paper/1904.00350/) (2019)

## Built on by (most cited first, 3 of 3)

- [Understanding the Therapeutic Relationship between Counselors and Clients in Online Text-based Counseling using LLMs](https://mlsanity.com/paper/2402.11958/) (2024)
- [Understanding Client Reactions in Online Mental Health Counseling](https://mlsanity.com/paper/2306.15334/) (2023)
- [Fine-Tuning Large Audio-Language Models with LoRA for Precise Temporal Localization of Prolonged Exposure Therapy Elements](https://mlsanity.com/paper/2506.09707/) (2025)

## Abstract (arXiv)

Automated real-time evaluation of counselor-client interaction is important for ensuring quality counseling but the rules are difficult to articulate. Recent advancements in machine learning methods show the possibility of learning such rules automatically. However, these methods often demand large scale and high quality counseling data, which are difficult to collect. To address this issue, we build an online counseling platform, which allows professional psychotherapists to provide free counseling services to those are in need. In exchange, we collect the counseling transcripts. Within a year of its operation, we manage to get one of the largest set of (675) transcripts of counseling sessions. To further leverage the valuable data we have, we label our dataset using both coarse- and fine-grained labels and use a set of pretraining techniques. In the end, we are able to achieve practically useful accuracy in both labeling system.