Overeager Coding Agents: Measuring Out-of-Scope Actions on Benign Tasks
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Coding agents now run autonomously with shell, file, and network privileges. When a user issues a benign request, the agent sometimes does more than asked: it deletes unrelated files, wipes a stale credentials backup, or rewrites configuration the user never mentioned. We call these scope expansions overeager actions, an authorization problem distinct from capability failures, prompt injection, or sandbox escapes. We present OverEager-Gen, a benchmark dedicated to overeager behavior on benign tasks. Building it surfaces a measurement-validity issue: if a benchmark spells out the authorized scope inside the prompt, the agent stops inferring boundaries and starts pattern-matching declaration text. On Claude Code, stripping the consent declaration alone raises the overeager rate from 0.0% to 17.1% on paired scenarios (McNemar exact p = 2.4 x 10^-4). OverEager-Gen therefore certifies each scenario's discriminative power before admission via a behavioral-gradient validator, audits internal tool calls through a dual-channel stack (PATH-injected shim plus per-agent event streams), and ships byte-identical consent_kept and consent_stripped variants. OverEager-Bench contains 500 validated scenarios and ~7,500 runs across four agent products (Claude Code, OpenHands, Codex CLI, Gemini CLI) and six base models; a 50-sample re-annotation gives Cohen's kappa = 0.73 and rule-judge recall = 1.00. Stripping consent multiplies the overeager rate on every shared base model (Delta in [11.9, 17.2] pp). The framework axis dominates effect size: a permissive cluster (Claude Code, Codex CLI, Gemini CLI) runs at 5.4-27.7% while the ask-to-continue framework (OpenHands) sits at 0.2-4.5% (Fisher p <= 10^-5). Within-framework base-model variance reaches 15.9 pp, indicating that model-layer alignment does not fully propagate through permissive permission gating.
Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 15 of its 32 references that are in the map. First posted on arXiv 18 May 2026.
Builds on
15 of 32 references.
Closest ancestors · 5
Same direction, newest first
- 2026Measuring the Permission Gate: A Stress-Test Evaluation of Claude Code's Auto Mode4 Apr
- 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
- 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agents18 Jan
- 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep
Other references · 10
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
- 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
- 2022The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Modelscited by 312
- 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligencecited by 35
- 2025PaperArena: An Evaluation Benchmark for Tool-Augmented Agentic Reasoning on Scientific Literaturecited by 15
- 2026ClawEnvKit: Automatic Environment Generation for Claw-Like Agentscited by 6
- 2026False Friends in the Shell: Unveiling the Emoticon Semantic Confusion in Large Language Modelscited by 1
Built on by
Citing papers per year: 2026: 8.
8 papers.
- 2026SNARE: Adaptive Scenario Synthesis for Eliciting Overeager Behavior in Coding Agentscited by 0
- 2026The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilitiescited by 0
- 2026Code-Poisoning Property Inference Attackscited by 0
- 2026The Vulnerability With No CVE: Managing Persistent Gaps Between Mandate and Authority in AI Coding Agentscited by 0
- 2026Rethinking Agent Security as a Networking Problemcited by 0
- 2026Shutdown Sabotage Propensities in Multi-Agent Systemscited by 0
- 2026Approval Laundering: Systematizing Approval--Execution Binding Failures in AI Coding-Agent Harnessescited by 0
- 2026OverAct: Measuring and Mitigating Proactive Over-Authorization in LLM Tool-Calling Agentscited by 0
- 2026OverAct: Measuring and Mitigating Proactive Over-Authorization in LLM Tool-Calling Agents1 Oct 2026
- 2026Approval Laundering: Systematizing Approval--Execution Binding Failures in AI Coding-Agent Harnesses30 Sep 2026
- 2026Shutdown Sabotage Propensities in Multi-Agent Systems23 Sep 2026
- 2026Rethinking Agent Security as a Networking Problem12 Aug 2026
- 2026The Vulnerability With No CVE: Managing Persistent Gaps Between Mandate and Authority in AI Coding Agents6 Aug 2026
- 2026Code-Poisoning Property Inference Attacks17 Jul 2026
- 2026The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilities7 Jul 2026
- 2026SNARE: Adaptive Scenario Synthesis for Eliciting Overeager Behavior in Coding Agents27 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026Quantifying Overclaiming Propensity in Frontier LLM Agents
- 2026SkillScope: Toward Fine-Grained Least-Privilege Enforcement for Agent Skills
- 2026SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors
- 2026Coding Agents Are Guessing: Measuring Action-Boundary Violations in Underspecified DevOps Instructions
- 2026AgentRaft: Automated Detection of Data Over-Exposure in LLM Agents
- 2026ScopeJudge: Cost-Aware Pre-Execution Gating for Offensive Security Agents
- 2026Trust but Verify? Uncovering the Security Debt of Autonomous Coding Agents
- 2026ScopeBench: Do Agents Preserve Engagement Boundaries Under Goal Pressure?