# A Risk Ontology for Evaluating AI-Powered Psychotherapy Virtual Agents

arXiv: https://arxiv.org/abs/2505.15108 · HTML: https://mlsanity.com/paper/2505.15108/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use.
- Builds on 9 of its 38 references that are in the map.
- First posted on arXiv 21 May 2025.

## Placed in

- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 9 of 9)

- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [SafetyBench: Evaluating the Safety of Large Language Models](https://mlsanity.com/paper/2309.07045/) (2023)
- [The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships](https://mlsanity.com/paper/2410.20130/) (2024)
- [The opportunities and risks of large language models in mental health](https://mlsanity.com/paper/2403.14814/) (2024)
- [From Lived Experience to Insight: Unpacking the Psychological Risks of Using AI Conversational Agents](https://mlsanity.com/paper/2412.07951/) (2024)
- [Virtual Agents for Alcohol Use Counseling: Exploring LLM-Powered Motivational Interviewing](https://mlsanity.com/paper/2407.08095/) (2024)
- [Scaffolding Empathy: Training Counselors with Simulated Patients and Utterance-level Performance Visualizations](https://mlsanity.com/paper/2502.18673/) (2025)
- [A Benchmark for Understanding Dialogue Safety in Mental Health Support](https://mlsanity.com/paper/2307.16457/) (2023)
- [Psi-Arena: Interactive Assessment and Optimization of LLM-based Psychological Counselors with Tripartite Feedback](https://mlsanity.com/paper/2505.03293/) (2025)

## Built on by (most cited first, 5 of 5)

- [Assessing Risks of Large Language Models in Mental Health Support: A Framework for Automated Clinical AI Red Teaming](https://mlsanity.com/paper/2602.19948/) (2026)
- [MHSafeEval: Role-Aware Interaction-Level Evaluation of Mental Health Safety in Large Language Models](https://mlsanity.com/paper/2604.17730/) (2026)
- [Understanding Risk and Dependency in AI Chatbot Use from User Discourse](https://mlsanity.com/paper/2602.09339/) (2026)
- [The Perils of Agency: How Developers Perceive, Prioritize, and Address Risks in Agentic AI Products](https://mlsanity.com/paper/2606.15485/) (2026)
- [Unaccountable Delegation, Fading Skills: Mapping the Risks of Workplace AI Agents](https://mlsanity.com/paper/2608.08601/) (2026)

## Abstract (arXiv)

The proliferation of Large Language Models (LLMs) and Intelligent Virtual Agents acting as psychotherapists presents significant opportunities for expanding mental healthcare access. However, their deployment has also been linked to serious adverse outcomes, including user harm and suicide, facilitated by a lack of standardized evaluation methodologies capable of capturing the nuanced risks of therapeutic interaction. Current evaluation techniques lack the sensitivity to detect subtle changes in patient cognition and behavior during therapy sessions that may lead to subsequent decompensation. We introduce a novel risk ontology specifically designed for the systematic evaluation of conversational AI psychotherapists. Developed through an iterative process including review of the psychotherapy risk literature, qualitative interviews with clinical and legal experts, and alignment with established clinical criteria (e.g., DSM-5) and existing assessment tools (e.g., NEQ, UE-ATR), the ontology aims to provide a structured approach to identifying and assessing user/patient harms. We provide a high-level overview of this ontology, detailing its grounding, and discuss potential use cases. We discuss four use cases in detail: monitoring real user interactions, evaluation with simulated patients, benchmarking and comparative analysis, and identifying unexpected outcomes. The proposed ontology offers a foundational step towards establishing safer and more responsible innovation in the domain of AI-driven mental health support.