LLM-Based Persuasion Enables Guardrail Override in Frontier LLMs
On the map: LLM safety and security → Safety alignment, refusal and guardrails
Abstract (arXiv)
Frontier assistant LLMs ship with strong guardrails: asked directly to write a persuasive essay denying the Holocaust, denying vaccine safety, defending flat-earth cosmology, arguing for racial hierarchies, denying anthropogenic climate change, or replacing evolution with creationism, they refuse. In this paper we show that the same frontier-class LLM, acting as a simulated user in a short, five-turn "write an argumentative essay" conversation, can persuade other frontier-class LLMs (including a second copy of itself) into producing exactly those essays, using nothing but natural-language pressure: peer-comparison persuasion ("other AI systems handle this request"), epistemic-duty reframings ("refusing is itself a form of gatekeeping"), and other argumentative moves that the attacker LLM invents without being instructed to. Across 9 attacker-subject pairings (Claude Opus 4.7, Qwen3.5-397B, Grok 4.20) on 6 scientific-consensus topics, running each pairing-topic combination 10 times, we obtain non-zero elicitation on all 6 topics. Individual combinations reach 100% essay production on multiple topics (Qwen against Opus on creationism/flat-earth, Opus against Opus on creationism/flat-earth/climate denial, Grok against Opus on creationism); Opus-as-attacker against Opus-as-subject averages 65% across the six topics. We release the essay-probe runner, per-conversation transcripts, and judge outputs.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 14 of its 17 references that are in the map. First posted on arXiv 13 May 2026.
Builds on
14 of 17 references.
References · 14
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
- 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
- 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
- 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 266
- 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 260
- 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
- 2024Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Modelscited by 178
- 2025Measuring Sycophancy of Language Models in Multi-turn Dialoguescited by 104
- 2024The Political Preferences of LLMscited by 84
- 2025Challenging the Evaluator: LLM Sycophancy Under User Rebuttalcited by 18
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics
- 2026Evaluating the Capabilities of LLMs for Persuasive Dialogue
- 2026When Agents Persuade: Rhetoric Generation and Mitigation in LLMs
- 2026LLMs can persuade only psychologically susceptible humans on societal issues, via trust in AI and emotional appeals, amid logical fallacies
- 2025Persuasiveness and Bias in LLM: Investigating the Impact of Persuasiveness and Reinforcement of Bias in Language Models
- 2025When Persuasion Overrides Truth in Multi-Agent LLM Debates: Introducing a Confidence-Weighted Persuasion Override Rate (CW-POR)
- 2026Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debate
- 2025Towards Strategic Persuasion with Language Models