Drift-Bench: Diagnosing Cooperative Breakdowns in LLM Agents under Input Faults via Multi-Turn Interaction
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
As Large Language Models transition to autonomous agents, user inputs frequently violate cooperative assumptions (e.g., implicit intent, missing parameters, false presuppositions, or ambiguous expressions), creating execution risks that text-only evaluations do not capture. Existing benchmarks typically assume well-specified instructions or restrict evaluation to text-only, single-turn clarification, and thus do not measure multi-turn disambiguation under grounded execution risk. We introduce Drift-Bench, the first diagnostic benchmark that evaluates agentic pragmatics under input faults through multi-turn clarification across state-oriented and service-oriented execution environments. Grounded in classical theories of communication, Drift-Bench provides a unified taxonomy of cooperative breakdowns and employs a persona-driven user simulator with the Rise evaluation protocol. Experiments show substantial performance drops under these faults, with clarification effectiveness varying across user personas and fault types. MethodName bridges clarification research and agent safety evaluation, enabling systematic diagnosis of failures that can lead to unsafe executions.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 14 of its 49 references that are in the map. First posted on arXiv 2 Feb 2026.
Builds on
14 of 49 references.
Closest ancestors · 1
Same direction, newest first
Other references · 13
- 2023GPT-4 Technical Reportcited by 18,132
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
- 2025UserBench: An Interactive Gym Environment for User-Centric Agentscited by 69
- 2025AgentRouter: A Knowledge-Graph-Guided LLM Router for Collaborative Multi-Agent Question Answeringcited by 29
- 2023Sources of Uncertainty in Supervised Machine Learning -- A Statisticians' Viewcited by 27
- 2025AutoData: A Multi-Agent System for Open Web Data Collectioncited by 25
- 2025CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answeringcited by 15
- 2025MAPRO: Recasting Multi-Agent Prompt Optimization as Maximum a Posteriori Inferencecited by 15
- 2025ClarifyMT-Bench: Benchmarking and Improving Multi-Turn Clarification for Conversational Large Language Modelscited by 12
1 more
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026Why Semantic Entropy Fails: Geometry-Aware and Calibrated Uncertainty for Policy Optimizationcited by 5
- 2026AI Alignment Breaks at the Edgecited by 3
- 2026SAGE: Answer-Conditioned Uncertainty Targets for Verbal Uncertainty Alignmentcited by 2
- 2026Optimizing What Policies Learn From: Recoverability-aware Rollout Intervention Learningcited by 2
- 2026The Workload-Router-Pool Architecture for LLM Inference Optimization: A Vision Paper from the vLLM Semantic Router Projectcited by 1
- 2026Confidence Laundering in Agent Systems: Why Uncertainty Needs a Latent Carriercited by 1
- 2026HoosierHelp: Benchmarking LLM Agents for Social Service Navigationcited by 0
- 2026Optimizing What Policies Learn From: Recoverability-aware Rollout Intervention Learning5 Aug 2026
- 2026HoosierHelp: Benchmarking LLM Agents for Social Service Navigation3 Jul 2026
- 2026Confidence Laundering in Agent Systems: Why Uncertainty Needs a Latent Carrier9 Jun 2026
- 2026SAGE: Answer-Conditioned Uncertainty Targets for Verbal Uncertainty Alignment9 Jun 2026
- 2026Why Semantic Entropy Fails: Geometry-Aware and Calibrated Uncertainty for Policy Optimization20 May 2026
- 2026The Workload-Router-Pool Architecture for LLM Inference Optimization: A Vision Paper from the vLLM Semantic Router Project22 Mar 2026
- 2026AI Alignment Breaks at the Edge23 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2026MAS-FIRE: Fault Injection and Reliability Evaluation for LLM-Based Multi-Agent Systems
- 2026When Users Change Their Minds: Measuring and Repairing Intent Drift in LLM Agents
- 2026Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents
- 2026AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities
- 2026PASTABench: Proactive Assessment of Sequential Trajectories for Agent Safety
- 2026AgentBoundary: Counterfactual Evaluation of Safety in Tool-Using LLM Agents
- 2026Operational Hallucination and Safety Drift in AI Agents
- 2026AgentPerfBench: A Benchmarking and Evaluation Suite for Inference Performance of Agentic LLMs