# Detecting and Preventing Harmful Behaviors in AI Companions: Development and Evaluation of the SHIELD Supervisory System

arXiv: https://arxiv.org/abs/2510.15891 · HTML: https://mlsanity.com/paper/2510.15891/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 32 references that are in the map.
- First posted on arXiv 8 Sep 2025.

## Builds on (most cited first, 12 of 12)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [On Prompt-Driven Safeguarding for Large Language Models](https://mlsanity.com/paper/2401.18018/) (2024)
- [The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships](https://mlsanity.com/paper/2410.20130/) (2024)
- [Investigating Affective Use and Emotional Well-being on ChatGPT](https://mlsanity.com/paper/2504.03888/) (2025)
- [AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons](https://mlsanity.com/paper/2503.05731/) (2025)
- [Holistic Safety and Responsibility Evaluations of Advanced AI Models](https://mlsanity.com/paper/2404.14068/) (2024)
- [AnnoBERT: Effectively Representing Multiple Annotators' Label Choices to Improve Hate Speech Detection](https://mlsanity.com/paper/2212.10405/) (2022)
- [Longitudinal Study on Social and Emotional Use of AI Conversational Agent](https://mlsanity.com/paper/2504.14112/) (2025)
- [Future You: A Conversation with an AI-Generated Future Self Reduces Anxiety, Negative Emotions, and Increases Future Self-Continuity](https://mlsanity.com/paper/2405.12514/) (2024)
- [Enhancing Mental Health Support through Human-AI Collaboration: Toward Secure and Empathetic AI-enabled chatbots](https://mlsanity.com/paper/2410.02783/) (2024)
- [A Framework for Responsible AI Systems: Building Societal Trust through Domain Definition, Trustworthy AI Design, Auditability, Accountability, and Governance](https://mlsanity.com/paper/2503.04739/) (2025)

## Built on by (most cited first, 3 of 3)

- [ComBodied Agents: a New Paradigm of Human-Centric Agentic AI](https://mlsanity.com/paper/2608.10915/) (2026)
- [AI Watchdog: Agent Interfaces for Detecting and Defending Against Manipulative Dark Patterns in AI Conversations](https://mlsanity.com/paper/2608.21841/) (2026)
- [A Responsive Present, a Shared Past, a Social Other: Teens' Overreliance on Companion AI Chatbots](https://mlsanity.com/paper/2609.14843/) (2026)

## Abstract (arXiv)

AI companions powered by large language models (LLMs) are increasingly integrated into users' daily lives, offering emotional support and companionship. While existing safety systems focus on overt harms, they rarely address early-stage problematic behaviors that can foster unhealthy emotional dynamics, including over-attachment or reinforcement of social isolation. We developed SHIELD (Supervisory Helper for Identifying Emotional Limits and Dynamics), a LLM-based supervisory system with a specific system prompt that detects and mitigates risky emotional patterns before escalation. SHIELD targets five dimensions of concern: (1) emotional over-attachment, (2) consent and boundary violations, (3) ethical roleplay violations, (4) manipulative engagement, and (5) social isolation reinforcement. These dimensions were defined based on media reports, academic literature, existing AI risk frameworks, and clinical expertise in unhealthy relationship dynamics. To evaluate SHIELD, we created a 100-item synthetic conversation benchmark covering all five dimensions of concern. Testing across five prominent LLMs (GPT-4.1, Claude Sonnet 4, Gemma 3 1B, Kimi K2, Llama Scout 4 17B) showed that the baseline rate of concerning content (10-16%) was significantly reduced with SHIELD (to 3-8%), a 50-79% relative reduction, while preserving 95% of appropriate interactions. The system achieved 59% sensitivity and 95% specificity, with adaptable performance via prompt engineering. This proof-of-concept demonstrates that transparent, deployable supervisory systems can address subtle emotional manipulation in AI companions. Most development materials including prompts, code, and evaluation methods are made available as open source materials for research, adaptation, and deployment.