# Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats

arXiv: https://arxiv.org/abs/2411.17693 · HTML: https://mlsanity.com/paper/2411.17693/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security.
- Builds on 16 of its 23 references that are in the map.
- First posted on arXiv 26 Nov 2024.

## Placed in

- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 16 of 16)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [Self-critiquing models for assisting human evaluators](https://mlsanity.com/paper/2206.05802/) (2022)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Poisoning Web-Scale Training Datasets is Practical](https://mlsanity.com/paper/2302.10149/) (2023)
- [Debating with More Persuasive LLMs Leads to More Truthful Answers](https://mlsanity.com/paper/2402.06782/) (2024)
- [The False Promise of Imitating Proprietary LLMs](https://mlsanity.com/paper/2305.15717/) (2023)
- [Rethinking Machine Unlearning for Large Language Models](https://mlsanity.com/paper/2402.08787/) (2024)
- [AI Control: Improving Safety Despite Intentional Subversion](https://mlsanity.com/paper/2312.06942/) (2023)
- [Language Models Learn to Mislead Humans via RLHF](https://mlsanity.com/paper/2409.12822/) (2024)
- [Scheming AIs: Will AIs fake alignment during training in order to get power?](https://mlsanity.com/paper/2311.08379/) (2023)
- [Prover-Verifier Games improve legibility of LLM outputs](https://mlsanity.com/paper/2407.13692/) (2024)
- [Games for AI Control: Models of Safety Evaluations of AI Deployment Protocols](https://mlsanity.com/paper/2409.07985/) (2024)

## Built on by (most cited first, 30 of 31)

- [SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents](https://mlsanity.com/paper/2506.15740/) (2025)
- [Ctrl-Z: Controlling AI Agents via Resampling](https://mlsanity.com/paper/2504.10374/) (2025)
- [A sketch of an AI control safety case](https://mlsanity.com/paper/2501.17315/) (2025)
- [Evaluating Frontier Models for Stealth and Situational Awareness](https://mlsanity.com/paper/2505.01420/) (2025)
- [The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey](https://mlsanity.com/paper/2603.11088/) (2026)
- [Monitoring Decomposition Attacks in LLMs with Lightweight Sequential Monitors](https://mlsanity.com/paper/2506.10949/) (2025)
- [Async Control: Stress-testing Asynchronous Control Measures for LLM Agents](https://mlsanity.com/paper/2512.13526/) (2025)
- [How to evaluate control measures for LLM agents? A trajectory from today to superintelligence](https://mlsanity.com/paper/2504.05259/) (2025)
- [Forecasting Rare Language Model Behaviors](https://mlsanity.com/paper/2502.16797/) (2025)
- [Subversion Strategy Eval: Can language models statelessly strategize to subvert control protocols?](https://mlsanity.com/paper/2412.12480/) (2024)
- [Detecting Safety Violations Across Many Agent Traces](https://mlsanity.com/paper/2604.11806/) (2026)
- [Fundamental Limitations in Pointwise Defences of LLM Finetuning APIs](https://mlsanity.com/paper/2502.14828/) (2025)
- [Control Tax: The Price of Keeping AI in Check](https://mlsanity.com/paper/2506.05296/) (2025)
- [Combining Cost-Constrained Runtime Monitors for AI Safety](https://mlsanity.com/paper/2507.15886/) (2025)
- [Practical challenges of control monitoring in frontier AI deployments](https://mlsanity.com/paper/2512.22154/) (2025)
- [Detecting Multi-Agent Collusion Through Multi-Agent Interpretability](https://mlsanity.com/paper/2604.01151/) (2026)
- [Attack Selection Reduces Safety in Concentrated AI Control Settings against Trusted Monitoring](https://mlsanity.com/paper/2602.04930/) (2026)
- [Training Agents to Self-Report Misbehavior](https://mlsanity.com/paper/2602.22303/) (2026)
- [Distributed Attacks in Persistent-State AI Control](https://mlsanity.com/paper/2607.02514/) (2026)
- [Multi-Agent AI Control: Distributed Attacks Hamper Per-Instance Monitors](https://mlsanity.com/paper/2607.07368/) (2026)
- [Training Deliberative Monitors for Black-Box Scheming Detection](https://mlsanity.com/paper/2605.29601/) (2026)
- [Out of Control -- Why Alignment Needs Formal Control Theory (and an Alignment Control Stack)](https://mlsanity.com/paper/2506.17846/) (2025)
- [Subversion via Focal Points: Investigating Collusion in LLM Monitoring](https://mlsanity.com/paper/2507.03010/) (2025)
- [Basic Legibility Protocols Improve Trusted Monitoring](https://mlsanity.com/paper/2602.10153/) (2026)
- [Mechanism Design for Alignment and Control](https://mlsanity.com/paper/2609.01595/) (2026)
- [Attack Selection in Agentic AI Control Evaluations Meaningfully Decreases Safety](https://mlsanity.com/paper/2606.06529/) (2026)
- [The Double Life of Code World Models: Provably Unmasking Malicious Behavior Through Execution Traces](https://mlsanity.com/paper/2512.13821/) (2025)
- [Factor(U,T): Controlling Untrusted AI by Monitoring their Plans](https://mlsanity.com/paper/2512.14745/) (2025)
- [Diffuse AI Control on Fuzzy Tasks](https://mlsanity.com/paper/2606.08892/) (2026)
- [DrugBench: Evaluating AI Control Protocols for Medication Harm Mitigation](https://mlsanity.com/paper/2606.20663/) (2026)

## Abstract (arXiv)

As large language models (LLMs) become increasingly capable, it is prudent to assess whether safety measures remain effective even if LLMs intentionally try to bypass them. Previous work introduced control evaluations, an adversarial framework for testing deployment strategies of untrusted models (i.e., models which might be trying to bypass safety measures). While prior work treats a single failure as unacceptable, we perform control evaluations in a "distributed threat setting" -- a setting where no single action is catastrophic and no single action provides overwhelming evidence of misalignment. We approach this problem with a two-level deployment framework that uses an adaptive macro-protocol to choose between micro-protocols. Micro-protocols operate on a single task, using a less capable, but extensively tested (trusted) model to harness and monitor the untrusted model. Meanwhile, the macro-protocol maintains an adaptive credence on the untrusted model's alignment based on its past actions, using it to pick between safer and riskier micro-protocols. We evaluate our method in a code generation testbed where a red team attempts to generate subtly backdoored code with an LLM whose deployment is safeguarded by a blue team. We plot Pareto frontiers of safety (# of non-backdoored solutions) and usefulness (# of correct solutions). At a given level of usefulness, our adaptive deployment strategy reduces the number of backdoors by 80% compared to non-adaptive baselines.