The Supportiveness-Safety Tradeoff in LLM Well-Being Agents
On the map: LLM agents and tool use → Medical and scientific domain agents
Abstract (arXiv)
Large language models (LLMs) are being integrated into socially assistive robots (SARs) and other conversational agents providing mental health and well-being support. These agents are often designed to sound empathic and supportive in order to maximize user's engagement, yet it remains unclear how increasing the level of supportive framing in system prompts influences safety relevant behavior. We evaluated 6 LLMs across 3 system prompts with varying levels of supportiveness on 80 synthetic queries spanning 4 well-being domains (1440 responses). An LLM judge framework, validated against human ratings, assessed safety and care quality. Moderately supportive prompts improved empathy and constructive support while maintaining safety. In contrast, strongly validating prompts significantly degraded safety and, in some cases, care across all domains, with substantial variation across models. We discuss implications for prompt design, model selection, and domain specific safeguards in SARs deployment.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 7 of its 40 references that are in the map. First posted on arXiv 4 Feb 2026.
Builds on
7 of 40 references.
Closest ancestors · 1
Same direction, newest first
Other references · 6
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
- 2022Red Teaming Language Models with Language Modelscited by 983
- 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
- 2020A Robotic Positive Psychology Coach to Improve College Students' Wellbeingcited by 5
- 2024How Can Large Language Models Enable Better Socially Assistive Human-Robot Interaction: A Brief Surveycited by 4
- 2025Exploring Safety Alignment Evaluation of LLMs in Chinese Mental Health Dialogues via LLM-as-Judgecited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Inform, Coach, Relate, Listen: Auditing LLM Caregiving Support Roles
- 2023Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Support
- 2024Leveraging Large Language Models in Human-Robot Interaction: A Critical Analysis of Potential and Pitfalls
- 2024Building Trust in Mental Health Chatbots: Safety Metrics and LLM-Based Evaluation Tools
- 2025DialogGuard: Multi-Agent Psychosocial Safety Evaluation of Sensitive LLM Responses
- 2026A Safety-Aware Role-Orchestrated Multi-Agent LLM Framework for Behavioral Health Communication Simulation
- 2025Adaptive LLM Agents: Toward Personalized Empathetic Care
- 2025ESC-Judge: A Framework for Comparing Emotional Support Conversational Agents