# Initial Risk Probing and Feasibility Testing of Glow: a Generative AI-Powered Dialectical Behavior Therapy Skills Coach for Substance Use Recovery and HIV Prevention

arXiv: https://arxiv.org/abs/2602.08121 · HTML: https://mlsanity.com/paper/2602.08121/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 64 references that are in the map.
- First posted on arXiv 8 Feb 2026.

## Builds on (most cited first, 11 of 11)

- [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021)
- [Cultural Bias and Cultural Alignment of Large Language Models](https://mlsanity.com/paper/2311.14096/) (2023)
- [Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers](https://mlsanity.com/paper/2504.18412/) (2025)
- [How Well Do LLMs Represent Values Across Cultures? Empirical Analysis of LLM Responses Based on Hofstede Cultural Dimensions](https://mlsanity.com/paper/2406.14805/) (2024)
- [Do Large Language Models Align with Core Mental Health Counseling Competencies?](https://mlsanity.com/paper/2410.22446/) (2024)
- [Challenges of Large Language Models for Mental Health Counseling](https://mlsanity.com/paper/2311.13857/) (2023)
- [Virtual Agents for Alcohol Use Counseling: Exploring LLM-Powered Motivational Interviewing](https://mlsanity.com/paper/2407.08095/) (2024)
- [MAGI: Multi-Agent Guided Interview for Psychiatric Assessment](https://mlsanity.com/paper/2504.18260/) (2025)
- [Applying and Evaluating Large Language Models in Mental Health Care: A Scoping Review of Human-Assessed Generative Tasks](https://mlsanity.com/paper/2408.11288/) (2024)
- [Mitigating Hallucinations Using Ensemble of Knowledge Graph and Vector Store in Large Language Models to Enhance Mental Health Support](https://mlsanity.com/paper/2410.10853/) (2024)
- [Ask ChatGPT: Caveats and Mitigations for Individual Users of AI Chatbots](https://mlsanity.com/paper/2508.10272/) (2025)

## Abstract (arXiv)

Background: HIV and substance use represent interacting epidemics with shared psychological drivers - impulsivity and maladaptive coping. Dialectical behavior therapy (DBT) targets these mechanisms but faces scalability challenges. Generative artificial intelligence (GenAI) offers potential for delivering personalized DBT coaching at scale, yet rapid development has outpaced safety infrastructure. Methods: We developed Glow, a GenAI-powered DBT skills coach delivering chain and solution analysis for individuals at risk for HIV and substance use. In partnership with a Los Angeles community health organization, we conducted usability testing with clinical staff (n=6) and individuals with lived experience (n=28). Using the Helpful, Honest, and Harmless (HHH) framework, we employed user-driven adversarial testing wherein participants identified target behaviors and generated contextually realistic risk probes. We evaluated safety performance across 37 risk probe interactions. Results: Glow appropriately handled 73% of risk probes, but performance varied by agent. The solution analysis agent demonstrated 90% appropriate handling versus 44% for the chain analysis agent. Safety failures clustered around encouraging substance use and normalizing harmful behaviors. The chain analysis agent fell into an "empathy trap," providing validation that reinforced maladaptive beliefs. Additionally, 27 instances of DBT skill misinformation were identified. Conclusions: This study provides the first systematic safety evaluation of GenAI-delivered DBT coaching for HIV and substance use risk reduction. Findings reveal vulnerabilities requiring mitigation before clinical trials. The HHH framework and user-driven adversarial testing offer replicable methods for evaluating GenAI mental health interventions.