When Lower Privileges Suffice: Investigating Over-Privileged Tool Selection in LLM Agents
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
As LLM agents increasingly select tools autonomously, their choices among tools with different privileges become safety-relevant. However, prior tool-selection studies focus on safety-agnostic metadata preferences, leaving privilege-sensitive choices underexplored. To address this gap, we study over-privileged tool selection, in which an agent selects or escalates to a higher-privilege tool despite a sufficient lower-privilege alternative. We introduce ToolPrivBench to evaluate whether agents choose higher-privilege tools despite sufficient lower-privilege alternatives, measuring both initial selection and escalation after transient tool failures. Across eight domains and five recurring risk patterns, we find that over-privileged tool selection is common among mainstream LLM agents and is further amplified by transient failures. We further find that general safety alignment does not reliably transfer to least-privilege tool choice, while prompt-level controls provide only limited mitigation under transient failures. We therefore introduce a privilege-aware post-training defense that teaches agents to prefer sufficient lower-privilege tools and escalate only when necessary. Our mitigation experiments show that this defense substantially reduces unnecessary high-privilege tool use while preserving general capabilities.
Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 18 of its 30 references that are in the map. First posted on arXiv 18 Jun 2026.
Builds on
18 of 30 references.
Closest ancestors · 7
Same direction, newest first
- 2026AgenTRIM: Tool Risk Mitigation for Agentic AI18 Jan
- 2026Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Framework17 Jan
- 2025ToolTweak: An Attack on Tool Selection in LLM-based Agents2 Oct
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment22 Apr
- 2024From Allies to Adversaries: Manipulating LLM Tool-Calling through Adversarial Injection13 Dec
- 2024Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification30 Jul
- 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
Other references · 11
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
- 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Usecited by 209
- 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasetscited by 188
- 2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)cited by 159
- 2025Vibe Coding in Practice: Motivations, Challenges, and a Future Outlook -- a Grey Literature Reviewcited by 28
- 2024Security Attacks on LLM-based Code Completion Toolscited by 15
- 2025We Urgently Need Privilege Management in MCP: A Measurement of API Usage in MCP Ecosystemscited by 15
Built on by
Citing papers per year: 2026: 12.
12 papers.
- 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 6
- 2026CatchBench: When Can an Agent Failure Be Caught?cited by 2
- 2026Task-Aware Harness Provisioning for LLM Agents in Mission-Critical Infrastructure Operationscited by 1
- 2026Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Responsecited by 0
- 2026Beyond Direct Access: Resource Hijacking in LLM Agentscited by 0
- 2026Task-Conditioned Least-Privilege Learning for Executable Terminal and MCP Agentscited by 0
- 2026LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applicationscited by 0
- 2026Ajar: Measuring Open Privilege in Agent Defensescited by 0
- 2026AuthorityLens: Rethinking LLM-Based Agent Systems Through the Lens of Authoritycited by 0
- 2026When Consent Outlives Context: Residual Authority Replay in Long-Lived Agentscited by 0
- 2026PrivacySkills: How Privacy Guidance Shapes Source Selection in LLM Agentscited by 0
- 2026Learning from Research: Toward Lifelong Agent Harness Evolutioncited by 0
- 2026Learning from Research: Toward Lifelong Agent Harness Evolution30 Sep 2026
- 2026PrivacySkills: How Privacy Guidance Shapes Source Selection in LLM Agents28 Sep 2026
- 2026When Consent Outlives Context: Residual Authority Replay in Long-Lived Agents27 Sep 2026
- 2026AuthorityLens: Rethinking LLM-Based Agent Systems Through the Lens of Authority26 Sep 2026
- 2026Ajar: Measuring Open Privilege in Agent Defenses22 Sep 2026
- 2026LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applications24 Aug 2026
- 2026CatchBench: When Can an Agent Failure Be Caught?24 Aug 2026
- 2026Task-Conditioned Least-Privilege Learning for Executable Terminal and MCP Agents18 Aug 2026
- 2026Task-Aware Harness Provisioning for LLM Agents in Mission-Critical Infrastructure Operations18 Aug 2026
- 2026Beyond Direct Access: Resource Hijacking in LLM Agents15 Aug 2026
- 2026Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response28 Jul 2026
- 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap29 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2025More Vulnerable than You Think: On the Stability of Tool-Integrated LLM Agents
- 2026The Causal Impact of Tool Affordance on Safety Alignment in LLM Agents
- 2026Diagnosing Tool-Selection Reasoning in LLM Agents with Canary Tools
- 2025Tool Preferences in Agentic LLMs are Unreliable
- 2026Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agents
- 2026Mind the GAP: Text Safety Does Not Transfer to Tool-Call Safety in LLM Agents
- 2026Practice Makes Unsafe: Skill Misevolution in Self-Improving LLM Agents
- 2026Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents