# Can You Share Your Story? Modeling Clients' Metacognition and Openness for LLM Therapist Evaluation

arXiv: https://arxiv.org/abs/2507.19643 · HTML: https://mlsanity.com/paper/2507.19643/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 22 references that are in the map.
- First posted on arXiv 25 Jul 2025.

## Builds on (most cited first, 10 of 10)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [A Computational Framework for Behavioral Assessment of LLM Therapists](https://mlsanity.com/paper/2401.00820/) (2024)
- [Cactus: Towards Psychological Counseling Conversations using Cognitive Behavioral Theory](https://mlsanity.com/paper/2407.03103/) (2024)
- [HealMe: Harnessing Cognitive Reframing in Large Language Models for Psychotherapy](https://mlsanity.com/paper/2403.05574/) (2024)
- [CBT-LLM: A Chinese Large Language Model for Cognitive Behavioral Therapy-based Mental Health Question Answering](https://mlsanity.com/paper/2403.16008/) (2024)
- [Interactive Agents: Simulating Counselor-Client Psychological Counseling via Role-Playing LLM-to-LLM Interactions](https://mlsanity.com/paper/2408.15787/) (2024)
- [Towards a Client-Centered Assessment of LLM Therapists by Client Simulation](https://mlsanity.com/paper/2406.12266/) (2024)
- [Understanding the Therapeutic Relationship between Counselors and Clients in Online Text-based Counseling using LLMs](https://mlsanity.com/paper/2402.11958/) (2024)
- [Rethinking Large Language Models in Mental Health Applications](https://mlsanity.com/paper/2311.11267/) (2023)
- [Multi-Session Client-Centered Treatment Outcome Evaluation in Psychotherapy](https://mlsanity.com/paper/2410.05824/) (2024)

## Built on by (most cited first, 7 of 7)

- [Narrative Theory-Driven LLM Methods for Automatic Story Generation and Understanding: A Survey](https://mlsanity.com/paper/2602.15851/) (2026)
- [PatientHub: A Unified Framework for Patient Simulation](https://mlsanity.com/paper/2602.11684/) (2026)
- [CCD-CBT: Multi-Agent Therapeutic Interaction for CBT Guided by Cognitive Conceptualization Diagram](https://mlsanity.com/paper/2604.06551/) (2026)
- [Metacognition in LLMs: Foundations, Progress, and Opportunities](https://mlsanity.com/paper/2607.11881/) (2026)
- [PatientAct: Theory-Grounded Mental Health Client Simulation](https://mlsanity.com/paper/2608.12750/) (2026)
- [Ask or Answer: A Decision Framework for Multi-Turn Health Misinformation Intervention](https://mlsanity.com/paper/2608.21721/) (2026)
- [From Pattern Recognizers to Personalized Companions: A Survey of Large Language Models in Mental Health](https://mlsanity.com/paper/2609.25186/) (2026)

## Abstract (arXiv)

Understanding clients' thoughts and beliefs is fundamental in counseling, yet current evaluations of LLM therapists often fail to assess this ability. Existing evaluation methods rely on client simulators that clearly disclose internal states to the therapist, making it difficult to determine whether an LLM therapist can uncover unexpressed perspectives. To address this limitation, we introduce MindVoyager, a novel evaluation framework featuring a controllable and realistic client simulator which dynamically adapts itself based on the ongoing counseling session, offering a more realistic and challenging evaluation environment. We further introduce evaluation metrics that assess the exploration ability of LLM therapists by measuring their thorough understanding of client's beliefs and thoughts.