# Beyond "I Can't Help With That": How Child Safety Experts Evaluate AI Chatbot Safety

arXiv: https://arxiv.org/abs/2608.07902 · HTML: https://mlsanity.com/paper/2608.07902/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 94 references that are in the map.
- First posted on arXiv 8 Aug 2026.

## Builds on (most cited first, 23 of 23)

- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal](https://mlsanity.com/paper/2406.14598/) (2024)
- [MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs](https://mlsanity.com/paper/2501.17399/) (2025)
- [Sociotechnical Safety Evaluation of Generative AI Systems](https://mlsanity.com/paper/2310.11986/) (2023)
- [Participation is not a Design Fix for Machine Learning](https://mlsanity.com/paper/2007.02423/) (2020)
- [The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice](https://mlsanity.com/paper/2310.00907/) (2023)
- [The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships](https://mlsanity.com/paper/2410.20130/) (2024)
- [Virtual Personas for Language Models via an Anthology of Backstories](https://mlsanity.com/paper/2407.06576/) (2024)
- [Participation in the age of foundation models](https://mlsanity.com/paper/2405.19479/) (2024)
- [ChaCha: Leveraging Large Language Models to Prompt Children to Share Their Emotions about Personal Events](https://mlsanity.com/paper/2309.12244/) (2023)
- [Taxonomizing and Measuring Representational Harms: A Look at Image Tagging](https://mlsanity.com/paper/2305.01776/) (2023)
- [Lessons From Red Teaming 100 Generative AI Products](https://mlsanity.com/paper/2501.07238/) (2025)
- ["Don't Forget the Teachers": Towards an Educator-Centered Understanding of Harms from Large Language Models in Education](https://mlsanity.com/paper/2502.14592/) (2025)
- [MinorBench: A hand-built benchmark for content-based risks for children](https://mlsanity.com/paper/2503.10242/) (2025)
- [Operationalizing Pluralistic Values in Large Language Model Alignment Reveals Trade-offs in Safety, Inclusivity, and Model Behavior](https://mlsanity.com/paper/2511.14476/) (2025)
- [Principles of Safe AI Companions for Youth: Parent and Expert Perspectives](https://mlsanity.com/paper/2510.11185/) (2025)
- [Understanding Gen Alpha Digital Language: Evaluation of LLM Safety Systems for Content Moderation](https://mlsanity.com/paper/2505.10588/) (2025)
- [SproutBench: A Benchmark for Safe and Ethical Large Language Models for Youth](https://mlsanity.com/paper/2508.11009/) (2025)
- [PluriHarms: Benchmarking the Full Spectrum of Human Judgments on AI Harm](https://mlsanity.com/paper/2601.08951/) (2026)
- [Measuring Machine Learning Harms from Stereotypes Requires Understanding Who Is Harmed by Which Errors in What Ways](https://mlsanity.com/paper/2402.04420/) (2024)
- [Expert Evaluation and the Limits of Human Feedback in Mental Health AI Safety Testing](https://mlsanity.com/paper/2601.18061/) (2026)
- [Understanding Parents' Desires in Moderating Children's Interactions with GenAI Chatbots through LLM-Generated Probes](https://mlsanity.com/paper/2603.03727/) (2026)
- [Whose Knowledge Counts? Co-Designing Community-Centered AI Auditing Tools with Educators in Hawai`i](https://mlsanity.com/paper/2603.16646/) (2026)

## Abstract (arXiv)

Youth increasingly turn to AI chatbots for social and emotional support, raising concerns about how these systems respond, especially in high-stakes situations. However, existing child safety evaluations of AI lack grounding in real-world harms that youth experience, rely on unvalidated assumptions about what counts as an appropriate output (e.g., refusal), and typically focus on detecting adversarial prompts or surface-level harms in outputs only. Thus, these evaluations can fail to detect responses that pose harm to youth in practice. To better understand the limitations of current evaluation practices, we conducted interviews with 19 practitioners working directly with youth in vulnerable situations, including social workers, therapists, and psychologists, asking them to reflect on chatbots' responses to risky situations commonly faced by youth, as established in prior empirical work. Practitioners identified chatbot behaviors likely to cause harm as well as those that could meaningfully support youth in difficult moments, discussed the role that chatbots should (and should not) play in these interactions, and offered concrete recommendations for improving chatbot responses. Based on these findings, we provide recommendations for AI child safety evaluation and infrastructure, and highlight the need for incorporating practitioners' perspectives into safety work.