Capability Minimization as a Safety Primitive: Risk-Aware Causal Gating for Least-Privilege LLM Agents
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Modern decision systems increasingly rely on learned components whose outputs may be confident yet wrong, exposing downstream actions to costly errors. We introduce Risk-Aware Causal Gating (RACG), a framework that decides whether to act on, defer, or abstain from a model's prediction by combining causal effect estimation with calibrated risk control. RACG models the causal pathway from candidate actions to outcomes and gates each decision according to an estimated counterfactual risk rather than raw predictive confidence. To make gating reliable, we derive distribution-free bounds on the probability of acting under high-risk conditions and show how these bounds translate into operating thresholds that satisfy user-specified safety constraints. We further propose an adaptive gating policy that adjusts to distribution shift by monitoring discrepancies between predicted and realized outcomes, tightening the gate when causal assumptions appear violated. Across simulated interventions and real-world decision benchmarks, RACG reduces high-cost errors substantially while preserving most of the utility of an ungated policy, and it outperforms confidence-based and selective-prediction baselines at matched abstention rates. Our results indicate that explicitly separating causal risk from predictive uncertainty yields decision systems that are both safer and more transparent, offering a principled mechanism for trustworthy automation in high-stakes settings.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 16 of its 24 references that are in the map. First posted on arXiv 11 Jun 2026.
Builds on
16 of 24 references.
References · 16
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
- 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 591
- 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 548
- 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 288
- 2025Retrieval Models Aren't Tool-Savvy: Benchmarking Tool Retrieval for Large Language Modelscited by 47
- 2023Testing Language Model Agents Safely in the Wildcited by 46
- 2025RAG-MCP: Mitigating Prompt Bloat in LLM Tool Selection via Retrieval-Augmented Generationcited by 28
- 2025ToolScope: Enhancing LLM Agent Tool Use through Tool Merging and Context-Aware Filteringcited by 19
4 more
- 2026ToolChoiceConfusion: Causal Minimal Tool Filtering for Reliable LLM Agentscited by 9
- 2026How Many Tools Should an LLM Agent See? A Chance-Corrected Answercited by 7
- 2026Self-Healing Agentic Orchestrators for Reliable Tool-Augmented Large Language Model Systemscited by 7
- 2026Contract2Tool: Learning Preconditions and Effects for Reliable Tool-Augmented LLM Agentscited by 5
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Lingering Authority: Revocable Resource-and-Effect Capabilities for Coding Agentscited by 8
- 2026GIST-CMTF: Goal-State Inference for Causal Minimal Tool Filtering in LLM Agentscited by 2
- 2026The Gate Is Only as Honest as Its Contracts: ContractGuard for the Contract Layer of Risk-Aware Causal Gatingcited by 2
- 2026A Benchmark and Diagnostic Study of Epistemic Admission in Shared Agent Memorycited by 1
- 2026Risk Architecture for AI-Native Engineering Teams: An Organizational Framework for Agentic System Governancecited by 0
- 2026Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Studycited by 0
- 2026A Benchmark and Diagnostic Study of Epistemic Admission in Shared Agent Memory25 Sep 2026
- 2026Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Study7 Sep 2026
- 2026Risk Architecture for AI-Native Engineering Teams: An Organizational Framework for Agentic System Governance1 Jul 2026
- 2026Lingering Authority: Revocable Resource-and-Effect Capabilities for Coding Agents21 Jun 2026
- 2026The Gate Is Only as Honest as Its Contracts: ContractGuard for the Contract Layer of Risk-Aware Causal Gating17 Jun 2026
- 2026GIST-CMTF: Goal-State Inference for Causal Minimal Tool Filtering in LLM Agents15 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026Calibration Is Not Control: Why LLM-Agent Oversight Needs Intervention
- 2025Safe But Not Sorry: Reducing Over-Conservatism in Safety Critics via Uncertainty-Aware Modulation
- 2026Same Signal, Opposite Meaning: Direction-Informed Adaptive Learning for LLM Agents
- 2025Causal Bandit Over Unknown Graphs: Upper Confidence Bounds With Backdoor Adjustment
- 2024TRAM: Test-Time Risk Adaptation with Mixture of Agents
- 2026MemRiskBench: Trace-Aware Risk-Preserving Evaluation for Long-Horizon LLM Agents
- 2026Action-Conditioned Risk Gating for Safety-Critical Control under Partial Observability
- 2026Hindsight-Anchored Policy Optimization: Learning Through Hindsight with Thompson Sampling-Inspired Adaptive Gating