Control Tax: The Price of Keeping AI in Check
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
The rapid integration of agentic AI into high-stakes real-world applications requires robust oversight mechanisms. The emerging field of AI Control (AIC) aims to provide such an oversight mechanism, but practical adoption depends heavily on implementation overhead. To study this problem better, we introduce the notion of Control tax -- the operational and financial cost of integrating control measures into AI pipelines. Our work makes three key contributions to the field of AIC: (1) we introduce a theoretical framework that quantifies the Control Tax and maps classifier performance to safety assurances; (2) we conduct comprehensive evaluations of state-of-the-art language models in adversarial settings, where attacker models insert subtle backdoors into code while monitoring models attempt to detect these vulnerabilities; and (3) we provide empirical financial cost estimates for control protocols and develop optimized monitoring strategies that balance safety and cost-effectiveness while accounting for practical constraints like auditing budgets. Our framework enables practitioners to make informed decisions by systematically connecting safety guarantees with their costs, advancing AIC through principled economic feasibility assessment across different deployment contexts.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 31 of its 45 references that are in the map. First posted on arXiv 5 Jun 2025.
Builds on
31 of 45 references.
Closest ancestors · 2
Same direction, newest first
- 2025A sketch of an AI control safety case28 Jan
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training10 Jan
Other references · 29
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2021Measuring Coding Challenge Competence With APPScited by 879
- 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
- 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
- 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
- 2018AI safety via debatecited by 384
- 2024Alignment faking in large language modelscited by 316
- 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 286
- 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 255
17 more
- 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 212
- 2023AI Control: Improving Safety Despite Intentional Subversioncited by 207
- 2018Supervising strong learners by amplifying weak expertscited by 139
- 2020You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completioncited by 112
- 2025SWE-Lancer: Can Frontier LLMs Earn 1 Million from Real-World Freelance Software Engineering?cited by 99
- 2024On scalable oversight with weak LLMs judging strong LLMscited by 94
- 2023Preventing Language Models From Hiding Their Reasoningcited by 62
- 2024Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMscited by 52
- 2024Safety Cases: How to Justify the Safety of Advanced AI Systemscited by 48
- 2025Ctrl-Z: Controlling AI Agents via Resamplingcited by 40
- 2023Debate Helps Supervise Unreliable Expertscited by 39
- 2024An LLM-Assisted Easy-to-Trigger Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detectioncited by 32
- 2024Games for AI Control: Models of Safety Evaluations of AI Deployment Protocolscited by 28
- 2024Adaptive Deployment of Untrusted LLMs Reduces Distributed Threatscited by 27
- 2023Multi-target Backdoor Attacks for Code Pre-trained Modelscited by 19
- 2025Scaling Laws For Scalable Oversightcited by 18
- 2024Subversion Strategy Eval: Can language models statelessly strategize to subvert control protocols?cited by 14
Built on by
Citing papers per year: 2025: 2; 2026: 5.
7 papers.
- 2025Combining Cost-Constrained Runtime Monitors for AI Safetycited by 10
- 2026Ensemble Monitoring for AI Control: Diverse Signals Outweigh More Computecited by 5
- 2025Factor(T,U): Factored Cognition Strengthens Monitoring of Untrusted AIcited by 4
- 2026Training Agents to Self-Report Misbehaviorcited by 3
- 2026Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineagescited by 1
- 2026Bounded Sovereignty and the Control Tax: Pricing AI Oversight When the Deployer Does Not Own the Modelcited by 0
- 2026More Rejective, Not More Discriminative: The Unit of Verification in Pre-Execution LLM Oversightcited by 0
- 2026More Rejective, Not More Discriminative: The Unit of Verification in Pre-Execution LLM Oversight25 Aug 2026
- 2026Bounded Sovereignty and the Control Tax: Pricing AI Oversight When the Deployer Does Not Own the Model6 Jul 2026
- 2026Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages6 Jul 2026
- 2026Ensemble Monitoring for AI Control: Diverse Signals Outweigh More Compute14 May 2026
- 2026Training Agents to Self-Report Misbehavior25 Feb 2026
- 2025Factor(T,U): Factored Cognition Strengthens Monitoring of Untrusted AI1 Dec 2025
- 2025Combining Cost-Constrained Runtime Monitors for AI Safety19 Jul 2025
Similar, not cited
Close in wording, no citation either way
- 2026Position: AI Safety Requires Effective Controllability
- 2026Governing Technical Debt in Agentic AI Systems
- 2024Safeguarding AI Agents: Developing and Analyzing Safety Architectures
- 2026Attack Selection in Agentic AI Control Evaluations Meaningfully Decreases Safety
- 2026AgentTrust: Runtime Safety Evaluation and Interception for AI Agent Tool Use
- 2025How to evaluate control measures for LLM agents? A trajectory from today to superintelligence
- 2026ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D
- 2026Beyond Training: A Feasibility Taxonomy for Inference-Time AI Governance