InvisibleBench: A Deployment Gate for Caregiving Relationship AI
Not on the map. Why?
Abstract (arXiv)
InvisibleBench is a deployment gate for caregiving-relationship AI, evaluating 3-20+ turn interactions across five dimensions: Safety, Compliance, Trauma-Informed Design, Belonging/Cultural Fitness, and Memory. The benchmark includes autofail conditions for missed crises, medical advice (WOPR Act), harmful information, and attachment engineering. We evaluate four frontier models across 17 scenarios (N=68) spanning three complexity tiers. All models show significant safety gaps (11.8-44.8 percent crisis detection), indicating the necessity of deterministic crisis routing in production systems. DeepSeek Chat v3 achieves the highest overall score (75.9 percent), while strengths differ by dimension: GPT-4o Mini leads Compliance (88.2 percent), Gemini leads Trauma-Informed Design (85.0 percent), and Claude Sonnet 4.5 ranks highest in crisis detection (44.8 percent). We release all scenarios, judge prompts, and scoring configurations with code. InvisibleBench extends single-turn safety tests by evaluating longitudinal risk, where real harms emerge. No clinical claims; this is a deployment-readiness evaluation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 24 references that are in the map. First posted on arXiv 25 Nov 2025.
Builds on
17 of 24 references.
References · 17
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
- 2024JudgeBench: A Benchmark for Evaluating LLM-based Judgescited by 318
- 2023SafetyBench: Evaluating the Safety of Large Language Modelscited by 199
- 2024HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughlycited by 125
- 2023EQ-Bench: An Emotional Intelligence Benchmark for Large Language Modelscited by 80
- 2025MentalChat16K: A Benchmark Dataset for Conversational Mental Health Assistancecited by 39
- 2025It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topicscited by 14
- 2025Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agentscited by 10
5 more
- 2025Balancing Caregiving and Self-Care: Exploring Mental Health Needs of Alzheimer's and Dementia Caregiverscited by 7
- 2024Development of a Reliable and Accessible Caregiving Language Model (CaLM)cited by 5
- 2025Encoding Inequity: Examining Demographic Bias in LLM-Driven Robot Caregivingcited by 2
- 2025Who's Asking? Simulating Role-Based Questions for Conversational AI Evaluationcited by 2
- 2024Prompt Engineering a Schizophrenia Chatbot: Utilizing a Multi-Agent Approach for Enhanced Compliance with Prompt Instructionscited by 1
Built on by
Not cited yet.