# LLM Can be a Dangerous Persuader: Empirical Study of Persuasion Safety in Large Language Models

arXiv: https://arxiv.org/abs/2504.10430 · HTML: https://mlsanity.com/paper/2504.10430/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 33 references that are in the map.
- First posted on arXiv 14 Apr 2025.

## Builds on (most cited first, 19 of 19)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs](https://mlsanity.com/paper/2401.06373/) (2024)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good](https://mlsanity.com/paper/1906.06725/) (2019)
- [The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation](https://mlsanity.com/paper/2312.09085/) (2023)
- [Combating Misinformation in the Age of LLMs: Opportunities and Challenges](https://mlsanity.com/paper/2311.05656/) (2023)
- [RTP-LX: Can LLMs Evaluate Toxicity in Multilingual Scenarios?](https://mlsanity.com/paper/2404.14397/) (2024)
- [Persuasion with Large Language Models: A Survey of Empirical Evidence, Study Methodologies, and Ethical Implications](https://mlsanity.com/paper/2411.06837/) (2024)
- [X-Eval: Generalizable Multi-aspect Text Evaluation via Augmented Instruction Tuning with Auxiliary Evaluation Aspects](https://mlsanity.com/paper/2311.08788/) (2023)
- [Holistic Evaluation for Interleaved Text-and-Image Generation](https://mlsanity.com/paper/2406.14643/) (2024)
- [Navigating LLM Ethics: Advancements, Challenges, and Future Directions](https://mlsanity.com/paper/2406.18841/) (2024)
- [How to Protect Yourself from 5G Radiation? Investigating LLM Responses to Implicit Misinformation](https://mlsanity.com/paper/2503.09598/) (2025)
- [Weakly-Supervised Hierarchical Models for Predicting Persuasive Strategies in Good-faith Textual Requests](https://mlsanity.com/paper/2101.06351/) (2021)
- [Toward Integrated Solutions: A Systematic Interdisciplinary Review of Cybergrooming Research](https://mlsanity.com/paper/2503.05727/) (2025)

## Built on by (most cited first, 23 of 23)

- [It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics](https://mlsanity.com/paper/2506.02873/) (2025)
- [Tree-based Dialogue Reinforced Policy Optimization for Red-Teaming Attacks](https://mlsanity.com/paper/2510.02286/) (2025)
- [Evaluating Language Models for Harmful Manipulation](https://mlsanity.com/paper/2603.25326/) (2026)
- [Exploring Consciousness in LLMs: A Systematic Survey of Theories, Implementations, and Frontier Risks](https://mlsanity.com/paper/2505.19806/) (2025)
- [Scheming Ability in LLM-to-LLM Strategic Interactions](https://mlsanity.com/paper/2510.12826/) (2025)
- [EpidemIQs: Prompt-to-Paper LLM Agents for Epidemic Modeling and Analysis](https://mlsanity.com/paper/2510.00024/) (2025)
- [Being Kind Isn't Always Being Safe: Diagnosing Affective Hallucination in LLMs](https://mlsanity.com/paper/2508.16921/) (2025)
- [GTAlign: Game-Theoretic Alignment of LLM Assistants for Social Welfare](https://mlsanity.com/paper/2510.08872/) (2025)
- [Breaking Minds, Breaking Systems: Jailbreaking Large Language Models via Human-like Psychological Manipulation](https://mlsanity.com/paper/2512.18244/) (2025)
- [The Hidden Puppet Master: Predicting Human Belief Change in Manipulative LLM Dialogues](https://mlsanity.com/paper/2603.20907/) (2026)
- [Ethics and Persuasion in Reinforcement Learning from Human Feedback: A Procedural Rhetorical Approach](https://mlsanity.com/paper/2505.09576/) (2025)
- [Seeing is Believing? Evaluating Vision-Language Model Susceptibility in Agent-to-Agent Multimodal Persuasion](https://mlsanity.com/paper/2510.22768/) (2025)
- [Persuasion Propagation: Does Persuasion Change AI Agent Behavior?](https://mlsanity.com/paper/2602.00851/) (2026)
- [SEVRA-BENCH: Social Engineering of Vulnerabilities in Review Agents](https://mlsanity.com/paper/2606.13757/) (2026)
- [Spontaneous Persuasion: An Audit of Model Persuasiveness in Everyday Conversations](https://mlsanity.com/paper/2604.22109/) (2026)
- [Assessing and Explaining the Persuadability of Large Language Models as Legal Decision Tools](https://mlsanity.com/paper/2604.26233/) (2026)
- [Just Ask for a Table: A Thirty-Token User Prompt Defeats Sponsored Recommendations in Twelve LLMs](https://mlsanity.com/paper/2605.12772/) (2026)
- [DECOR: Auditing LLM Deception via Information Manipulation Theory](https://mlsanity.com/paper/2605.19270/) (2026)
- [A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing](https://mlsanity.com/paper/2606.05330/) (2026)
- [Pareto-Guided Teacher Alignment for Fair Personalized Text Generation](https://mlsanity.com/paper/2606.10126/) (2026)
- [ALIBI: Adaptive Agentic Attacks on LLM-Based Vulnerability Detectors via Adversarial Code Comments](https://mlsanity.com/paper/2607.24964/) (2026)
- [A Multi-Stage Agentic Framework for Effective Counter-Narrative Generation and Refinement](https://mlsanity.com/paper/2609.14178/) (2026)
- [Understanding and Reducing Misalignment in Persuasion Optimization](https://mlsanity.com/paper/2610.16070/) (2026)

## Abstract (arXiv)

Recent advancements in Large Language Models (LLMs) have enabled them to approach human-level persuasion capabilities. However, such potential also raises concerns about the safety risks of LLM-driven persuasion, particularly their potential for unethical influence through manipulation, deception, exploitation of vulnerabilities, and many other harmful tactics. In this work, we present a systematic investigation of LLM persuasion safety through two critical aspects: (1) whether LLMs appropriately reject unethical persuasion tasks and avoid unethical strategies during execution, including cases where the initial persuasion goal appears ethically neutral, and (2) how influencing factors like personality traits and external pressures affect their behavior. To this end, we introduce PersuSafety, the first comprehensive framework for the assessment of persuasion safety which consists of three stages, i.e., persuasion scene creation, persuasive conversation simulation, and persuasion safety assessment. PersuSafety covers 6 diverse unethical persuasion topics and 15 common unethical strategies. Through extensive experiments across 8 widely used LLMs, we observe significant safety concerns in most LLMs, including failing to identify harmful persuasion tasks and leveraging various unethical persuasion strategies. Our study calls for more attention to improve safety alignment in progressive and goal-driven conversations such as persuasion.