# AICompanionBench: Benchmarking LLMs-as-Judges for AI Companion Safety

arXiv: https://arxiv.org/abs/2606.04867 · HTML: https://mlsanity.com/paper/2606.04867/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 61 references that are in the map.
- First posted on arXiv 3 Jun 2026.

## Builds on (most cited first, 12 of 12)

- [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://mlsanity.com/paper/2412.05579/) (2024)
- [R-Judge: Benchmarking Safety Risk Awareness for LLM Agents](https://mlsanity.com/paper/2401.10019/) (2024)
- [SafetyBench: Evaluating the Safety of Large Language Models](https://mlsanity.com/paper/2309.07045/) (2023)
- [GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysis](https://mlsanity.com/paper/2402.13494/) (2024)
- [AgentAuditor: Human-Level Safety and Security Evaluation for LLM Agents](https://mlsanity.com/paper/2506.00641/) (2025)
- [The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships](https://mlsanity.com/paper/2410.20130/) (2024)
- [AI Companions Reduce Loneliness](https://mlsanity.com/paper/2407.19096/) (2024)
- [Detecting Conversational Mental Manipulation with Intent-Aware Prompting](https://mlsanity.com/paper/2412.08414/) (2024)
- [HarmMetric Eval: Benchmarking Metrics and Judges for LLM Harmfulness Assessment](https://mlsanity.com/paper/2509.24384/) (2025)
- [Understanding Teen Overreliance on AI Companion Chatbots Through Self-Reported Reddit Narratives](https://mlsanity.com/paper/2507.15783/) (2025)
- [Principles of Safe AI Companions for Youth: Parent and Expert Perspectives](https://mlsanity.com/paper/2510.11185/) (2025)
- [The Rise of AI Companions: Interaction with AI Companions and Psychological Well-being](https://mlsanity.com/paper/2506.12605/) (2025)

## Built on by (most cited first, 1 of 1)

- [CompanionSafe: From Multi-turn Safety Evaluation to Guardrails for AI Companions](https://mlsanity.com/paper/2610.18095/) (2026)

## Abstract (arXiv)

As AI companion platforms such as Replika and Character.AI rapidly grow, concerns about unsafe human-AI interactions have intensified. This study introduces AICompanionBench, to our knowledge the first publicly available benchmark dataset of human-AI companion conversations annotated with fine-grained safety risk categories. The dataset contains 2,123 real-world Replika conversations collected from Reddit and annotated through human-AI collaboration across nine categories: sexual behavior, antisocial behavior, physical aggression, verbal aggression, substance abuse, self-harm and suicide, control, manipulation, and no-harm. Using this benchmark, we evaluate 20 state-of-the-art open-source and closed-source LLMs under an LLM-as-judge framework for detecting unsafe interactions. Results show substantial variation in model performance, with stronger models achieving high overall accuracy but still struggling with nuanced categories such as manipulation, as well as benign conversations that are incorrectly identified as harmful. Our findings suggest that while current LLMs can effectively detect explicit harmful content, they remain limited in identifying implicit unsafe interactions. Overall, our work contributes a new benchmark dataset for AI companionship safety research and offers insights into monitoring AI companion systems using LLMs. The dataset is publicly available at: https://github.com/anonymousresearcher2026/AICompanionBench/blob/main/AICompanionBench.xlsx