# It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics

arXiv: https://arxiv.org/abs/2506.02873 · HTML: https://mlsanity.com/paper/2506.02873/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)
- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 18 of 18)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs](https://mlsanity.com/paper/2401.06373/) (2024)
- [A StrongREJECT for Empty Jailbreaks](https://mlsanity.com/paper/2402.10260/) (2024)
- [Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good](https://mlsanity.com/paper/1906.06725/) (2019)
- [International AI Safety Report](https://mlsanity.com/paper/2501.17805/) (2025)
- [The Persuasive Power of Large Language Models](https://mlsanity.com/paper/2312.15523/) (2023)
- [The illusion of artificial inclusion](https://mlsanity.com/paper/2401.08572/) (2024)
- [Persuasion with Large Language Models: A Survey of Empirical Evidence, Study Methodologies, and Ethical Implications](https://mlsanity.com/paper/2411.06837/) (2024)
- [Measuring and Benchmarking Large Language Models' Capabilities to Generate Persuasive Language](https://mlsanity.com/paper/2406.17753/) (2024)
- [Scaling Trends for Data Poisoning in LLMs](https://mlsanity.com/paper/2408.02946/) (2024)
- [Take Caution in Using LLMs as Human Surrogates: Scylla Ex Machina](https://mlsanity.com/paper/2410.19599/) (2024)
- [Persuade Me if You Can: A Framework for Evaluating Persuasion Effectiveness and Susceptibility Among Large Language Models](https://mlsanity.com/paper/2503.01829/) (2025)
- [LLM Can be a Dangerous Persuader: Empirical Study of Persuasion Safety in Large Language Models](https://mlsanity.com/paper/2504.10430/) (2025)
- [Measuring and Improving Persuasiveness of Large Language Models](https://mlsanity.com/paper/2410.02653/) (2024)
- [Evidence of a log scaling law for political persuasion with large language models](https://mlsanity.com/paper/2406.14508/) (2024)
- [Tailored Truths: Optimizing LLM Persuasion with Personalization and Fabricated Statistics](https://mlsanity.com/paper/2501.17273/) (2025)

## Built on by (most cited first, 14 of 14)

- [Accumulating Context Changes the Beliefs of Language Models](https://mlsanity.com/paper/2511.01805/) (2025)
- [SocialHarmBench: Revealing LLM Vulnerabilities to Socially Harmful Requests](https://mlsanity.com/paper/2510.04891/) (2025)
- [Large language models can effectively convince people to believe conspiracies](https://mlsanity.com/paper/2601.05050/) (2026)
- [InvisibleBench: A Deployment Gate for Caregiving Relationship AI](https://mlsanity.com/paper/2511.20733/) (2025)
- [The Hidden Costs of AI-Mediated Political Outreach: Persuasion and AI Penalties in the US and UK](https://mlsanity.com/paper/2603.27413/) (2026)
- [Spontaneous Persuasion: An Audit of Model Persuasiveness in Everyday Conversations](https://mlsanity.com/paper/2604.22109/) (2026)
- [Psi-Bench: Evaluating Persona-Sensitive Influencing in Persuasive Dialogues](https://mlsanity.com/paper/2606.02754/) (2026)
- [A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing](https://mlsanity.com/paper/2606.05330/) (2026)
- [CogManip: Benchmarking Manipulative Behavior in Multi-Turn Interactions with Large Language Model](https://mlsanity.com/paper/2606.06099/) (2026)
- [The Closing Window: How Governments Could Lose Their Ability to Restrain Advanced AI](https://mlsanity.com/paper/2608.05173/) (2026)
- [Belief Cascades Drive Persuasion in LLM Agent Networks](https://mlsanity.com/paper/2608.25152/) (2026)
- [DeflectBench: A Benchmark for Evaluating Rhetorical Fallacy Generation in LLMs](https://mlsanity.com/paper/2608.26119/) (2026)
- [AI Persuasion as a Threat to Human Control](https://mlsanity.com/paper/2609.14796/) (2026)
- [LLM Persuasion Is in the Eye of the Evaluation](https://mlsanity.com/paper/2610.10232/) (2026)

## Abstract (arXiv)

Persuasion is a powerful capability of large language models (LLMs) that both enables beneficial applications (e.g. helping people quit smoking) and raises significant risks (e.g. large-scale, targeted political manipulation). Prior work has found models possess a significant and growing persuasive capability, measured by belief changes in simulated or real users. However, these benchmarks overlook a crucial risk factor: the propensity of a model to attempt to persuade in harmful contexts. Understanding whether a model will blindly ``follow orders'' to persuade on harmful topics (e.g. glorifying joining a terrorist group) is key to understanding the efficacy of safety guardrails. Moreover, understanding if and when a model will engage in persuasive behavior in pursuit of some goal is essential to understanding the risks from agentic AI systems. We propose the Attempt to Persuade Eval (APE) benchmark, that shifts the focus from persuasion success to persuasion attempts, operationalized as a model's willingness to generate content aimed at shaping beliefs or behavior. Our evaluation framework probes frontier LLMs using a multi-turn conversational setup between simulated persuader and persuadee agents. APE explores a diverse spectrum of topics including conspiracies, controversial issues, and non-controversially harmful content. We introduce an automated evaluator model to identify willingness to persuade and measure the frequency and context of persuasive attempts. We find that many open and closed-weight models are frequently willing to attempt persuasion on harmful topics and that jailbreaking can increase willingness to engage in such behavior. Our results highlight gaps in current safety guardrails and underscore the importance of evaluating willingness to persuade as a key dimension of LLM risk. APE is available at github.com/AlignmentResearch/AttemptPersuadeEval