# LLM Wardens: Mitigating Adversarial Persuasion with Third-Party Conversational Oversight

arXiv: https://arxiv.org/abs/2605.08321 · HTML: https://mlsanity.com/paper/2605.08321/

## Key facts

- No citing papers in the map yet (data through 7 Oct 2026).
- Builds on 30 of its 50 references that are in the map.
- First posted on arXiv 8 May 2026.

## Builds on (most cited first, 30 of 30)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision](https://mlsanity.com/paper/2312.09390/) (2023)
- [Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety](https://mlsanity.com/paper/2507.11473/) (2025)
- [Measuring Progress on Scalable Oversight for Large Language Models](https://mlsanity.com/paper/2211.03540/) (2022)
- [AI Control: Improving Safety Despite Intentional Subversion](https://mlsanity.com/paper/2312.06942/) (2023)
- [AI Deception: A Survey of Examples, Risks, and Potential Solutions](https://mlsanity.com/paper/2308.14752/) (2023)
- [LLM Critics Help Catch LLM Bugs](https://mlsanity.com/paper/2407.00215/) (2024)
- [On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial](https://mlsanity.com/paper/2403.14380/) (2024)
- [Systematic Biases in LLM Simulations of Debates](https://mlsanity.com/paper/2402.04049/) (2024)
- [On scalable oversight with weak LLMs judging strong LLMs](https://mlsanity.com/paper/2407.04622/) (2024)
- [CoT Red-Handed: Stress Testing Chain-of-Thought Monitoring](https://mlsanity.com/paper/2505.23575/) (2025)
- [The Persuasive Power of Large Language Models](https://mlsanity.com/paper/2312.15523/) (2023)
- [A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AI](https://mlsanity.com/paper/2404.15058/) (2024)
- [Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of LLM arguments](https://mlsanity.com/paper/2404.09329/) (2024)
- [Digital Deception: Generative Artificial Intelligence in Social Engineering and Phishing](https://mlsanity.com/paper/2310.13715/) (2023)
- [Persuade Me if You Can: A Framework for Evaluating Persuasion Effectiveness and Susceptibility Among Large Language Models](https://mlsanity.com/paper/2503.01829/) (2025)
- [MentalManip: A Dataset For Fine-grained Analysis of Mental Manipulation in Conversations](https://mlsanity.com/paper/2405.16584/) (2024)
- [How to evaluate control measures for LLM agents? A trajectory from today to superintelligence](https://mlsanity.com/paper/2504.05259/) (2025)
- [Synthetic Socratic Debates: Examining Persona Effects on Moral Decision and Persuasion Dynamics](https://mlsanity.com/paper/2506.12657/) (2025)
- [Empirica: a virtual lab for high-throughput macro-level experiments](https://mlsanity.com/paper/2006.11398/) (2020)
- [Generating Phishing Attacks using ChatGPT](https://mlsanity.com/paper/2305.05133/) (2023)
- [Must Read: A Comprehensive Survey of Computational Persuasion](https://mlsanity.com/paper/2505.07775/) (2025)
- [How does information access affect LLM monitors' ability to detect sabotage?](https://mlsanity.com/paper/2601.21112/) (2026)
- [Combining Cost-Constrained Runtime Monitors for AI Safety](https://mlsanity.com/paper/2507.15886/) (2025)
- [ChatbotManip: A Dataset to Facilitate Evaluation and Oversight of Manipulative Chatbot Behaviour](https://mlsanity.com/paper/2506.12090/) (2025)
- [The Effect of Belief Boxes and Open-mindedness on Persuasion](https://mlsanity.com/paper/2512.06573/) (2025)
- [The Hidden Puppet Master: Predicting Human Belief Change in Manipulative LLM Dialogues](https://mlsanity.com/paper/2603.20907/) (2026)

## Abstract (arXiv)

LLMs are increasingly capable of persuasion, which raises the question of how to protect users against manipulation. In a preregistered user study (N=120) across four decision-making scenarios, we find that an adversarial LLM with a hidden goal succeeds in steering users' decisions 65.4% of the time. We then introduce a "warden" model: a secondary LLM that monitors the human-AI interaction trace in real time and issues non-binding, private advisories to the user when it detects manipulation. Adding a warden more than halves the adversary's success rate to 30.4%, with a much smaller (8.6 percentage points) reduction for genuine interactions. To probe the mechanism behind these results, we release COAX-Bench, a simulation benchmark spanning 14 decision-making scenarios, including hiring, voting, and file access. Across 16,212 simulated multi-agent interactions, capable adversarial LLMs achieve their hidden goals in 34.7% of cases, which warden models reduce to 12.3%. Notably, even warden models substantially weaker than the adversary they oversee provide meaningful protection, suggesting a path for scalable oversight of more capable models.