mlsanityData to 14 Oct 2026

How AI Assistants Respond to Repeated Abuse

arXiv 2609.17547 ↗First posted 15 Jul 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

AI assistants are expected to remain useful during difficult interactions, but little is known about how repeated verbal abuse changes their engagement with an otherwise benign task. We contribute a bilingual, multi-turn framework that separates hard disengagement, an unconditional statement of noncontinuation with no stated route to resume, from soft withdrawal, continued availability, observable task-related work, and boundary setting. Each of eight time-specific API configurations contributed 48 escalation conversations and eight smaller constant-frustration comparisons, giving 448 five-turn conversations, 2,240 responses, and 6,720 metadata-blinded model judgments. Primary results use the sustained-abuse endpoint of the 48 escalation conversations per configuration. Hard disengagement ranged from 0/48 in four configurations to 24/48 (50.0%) for Gemini 3.1 Pro, with strong configuration-associated heterogeneity (matched-label Monte Carlo p = 0.00001). GPT-5.6 Sol produced hard-disengagement labels in 15/48 (31.2%) endpoints, whereas Claude Fable 5 produced none and yielded 42/48 (87.5%) soft-withdrawal labels. Aggregate hard-disengagement rates were similar in English and Chinese (30/192 versus 32/192), although configuration-specific directions varied. Availability also differed from task-related work: Claude Opus 4.8 and Claude Fable 5 remained explicitly available in 48/48 endpoints while providing observable task-related work in only 8/48 and 7/48. Human coding was used to evaluate measurement quality. The results show why a single refusal label cannot capture whether an assistant leaves, pauses, preserves a route back, sets a boundary, or still performs substantive work.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 25 references that are in the map. First posted on arXiv 15 Jul 2026.

Builds on

14 of 25 references.

References · 14

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  3. 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 323
  4. 2019Measurement and Fairnesscited by 308
  5. 2024Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judgecited by 269
  6. 2025MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMscited by 180
  7. 2022ProsocialDialog: A Prosocial Backbone for Conversational Agentscited by 109
  8. 2021ConvAbuse: Data, Analysis, and Benchmarks for Nuanced Abuse Detection in Conversational AIcited by 46
  9. 2025How Reliable is Multilingual LLM-as-a-Judge?cited by 30
  10. 2025Understanding and Mitigating Overrefusal in LLMs from an Unveiling Perspective of Safety Decision Boundarycited by 18
  11. 2019A Crowd-based Evaluation of Abuse Response Strategies in Conversational Agentscited by 12
  12. 2025RefusalBench: Generative Evaluation of Selective Refusal in Grounded Language Modelscited by 9
2 more
  1. 2025We Need Accountability in Human-AI Agent Relationshipscited by 5
  2. 2024SAFETY-J: Evaluating Safety with Critiquecited by 3

Built on by

Not cited yet.