Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigation
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
LLM agents can combine individually non-revealing tool returns and disclose a sensitive conclusion, creating Tools Orchestration Privacy Risk (TOP-R). We formalize TOP-R through three conditions: conclusion sensitivity, single-source non-inferability, and compositional inferability. We introduce Library-Grounded Reverse-Inference Seed Expansion (LRSE), a four-library reverse-construction pipeline, and use it to build TOP-Bench, a 1,000-instance benchmark evaluated under a controlled two-stage tool-use protocol. Across six LLM agents, average task completion, leakage, and H-score are 98.0 percent, 88.6 percent, and 20.4. With native reasoning enabled, four models average 81.4 percent final-response leakage and 82.4 percent reasoning-trace leakage. With reasoning disabled, three prompt-only safeguards improve H-score by an average of about 3.4 points on TOP-Bench. We further propose TOP-Align, an SFT+DPO method for learning safer task-completion boundaries. On a separate post-training evaluation set, TOP-Align improves H-score by 16.2 points over the base model, versus a 5.0-point average gain from prompt-only mitigation on the same set. These results show that TOP-R requires defenses beyond prompting alone. Dataset and code are available at https://github.com/1Ponder/TOP-R.
Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 52 of its 97 references that are in the map. First posted on arXiv 18 Dec 2025.
Builds on
52 of 97 references.
Closest ancestors · 11
Same direction, newest first
- 2025MAGPIE: A benchmark for Multi-AGent contextual PrIvacy Evaluation16 Oct
- 2025Exposing LLM User Privacy via Traffic Fingerprint Analysis: A Study of Privacy Risks in LLM Agent Interactions8 Oct
- 2025Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM-Powered Agents22 Sep
- 2025The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration16 Sep
- 2025Searching for Privacy Risks in LLM Agents via Simulation14 Aug
- 2025A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents30 Jun
- 2025Beyond Jailbreaking: Auditing Contextual Privacy in LLM Agents11 Jun
- 2025AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents12 Mar
- 2025Unveiling Privacy Risks in LLM Agent Memory17 Feb
- 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents14 Nov
- 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep
Other references · 41
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
- 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
- 2016Deep Learning with Differential Privacycited by 2,728
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
29 more
- 2020Extracting Training Data from Large Language Modelscited by 2,189
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
- 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
- 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
- 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 598
- 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 467
- 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 353
- 2023Tool Learning with Foundation Modelscited by 302
- 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 285
- 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
- 2023Large Language Model Guided Tree-of-Thoughtcited by 221
- 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 175
- 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 171
- 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 169
- 2024GoldCoin: Grounding Large Language Models in Privacy Laws via Contextual Integrity Theorycited by 37
- 2024CI-Bench: Benchmarking Contextual Integrity of AI Assistants on Synthetic Datacited by 37
- 2025Contextual Integrity in LLMs via Reasoning and Reinforcement Learningcited by 31
- 2025ToolMind Technical Report: A Large-Scale, Reasoning-Enhanced Tool-Use Datasetcited by 30
- 2025Leaky Thoughts: Large Reasoning Models Are Not Private Thinkerscited by 28
- 2025SoK: The Privacy Paradox of Large Language Models: Advancements, Privacy Risks, and Mitigationcited by 19
- 2025Tool Unlearning for Tool-Augmented LLMscited by 17
- 2025Position: Privacy Is Not Just Memorization!cited by 17
- 2024Rescriber: Smaller-LLM-Powered User-Led Data Minimization for LLM-Based Chatbotscited by 16
- 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
- 2025Evaluating Language Model Reasoning about Confidential Informationcited by 3
- 2025Correlation or Causation: Analyzing the Causal Structures of LLM and LRM Reasoning Processcited by 3
Built on by
Citing papers per year: 2026: 11.
11 papers.
- 2026AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systemscited by 25
- 2026When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystemscited by 17
- 2026AgentAbstain: Do LLM Agents Know When Not to Act?cited by 17
- 2026SUDP: Secret-Use Delegation Protocol for Agentic Systemscited by 2
- 2026PrivacyAlign: Contextual Privacy Alignment for LLM Agentscited by 2
- 2026Policy-Invisible Violations in LLM-Based Agentscited by 1
- 2026MCPHunt: An Evaluation Framework for Cross-Boundary Data Propagation in Multi-Server MCP Agentscited by 1
- 2026TRACE: Trajectory Risk-Aware Compression for Long-Horizon Agent Safetycited by 1
- 2026The Interlocutor Effect: Why LLMs Leak More Personal Data to Agents Than Humanscited by 1
- 2026MosaicLeaks:Privacy Risks in Querying-in-the-Open for Deep Research Agentscited by 0
- 2026Understanding and mitigating the risks of OpenClaw for non-technical users: A practical guide with Skillcited by 0
- 2026AgentAbstain: Do LLM Agents Know When Not to Act?11 Jul 2026
- 2026PrivacyAlign: Contextual Privacy Alignment for LLM Agents19 Jun 2026
- 2026Understanding and mitigating the risks of OpenClaw for non-technical users: A practical guide with Skill9 Jun 2026
- 2026TRACE: Trajectory Risk-Aware Compression for Long-Horizon Agent Safety30 May 2026
- 2026When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems30 May 2026
- 2026MosaicLeaks:Privacy Risks in Querying-in-the-Open for Deep Research Agents29 May 2026
- 2026MCPHunt: An Evaluation Framework for Cross-Boundary Data Propagation in Multi-Server MCP Agents30 Apr 2026
- 2026SUDP: Secret-Use Delegation Protocol for Agentic Systems27 Apr 2026
- 2026The Interlocutor Effect: Why LLMs Leak More Personal Data to Agents Than Humans26 Apr 2026
- 2026Policy-Invisible Violations in LLM-Based Agents14 Apr 2026
- 2026AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems12 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2026AgentRaft: Automated Detection of Data Over-Exposure in LLM Agents
- 2026ToolPrivacyBench: Benchmarking Purpose-Bound Privacy in Tool-Using LLM Agents
- 2025SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks
- 2026K-Bench: A Benchmark for LLM Unlearning in Agentic Deployments
- 2026When Lower Privileges Suffice: Investigating Over-Privileged Tool Selection in LLM Agents
- 2026SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents
- 2026Observable Channels, Not Just Storage: Evaluating Privacy Leakage in LLM Agent Pipelines
- 2026AgentCIScope: Grounded Contextual Integrity Evaluation for LLM Agents