# Can AI-Generated Persuasion Be Detected? Persuaficial Benchmark and AI vs. Human Linguistic Differences

arXiv: https://arxiv.org/abs/2601.04925 · HTML: https://mlsanity.com/paper/2601.04925/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 71 references that are in the map.
- First posted on arXiv 8 Jan 2026.

## Builds on (most cited first, 18 of 18)

- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good](https://mlsanity.com/paper/1906.06725/) (2019)
- [Can LLM-Generated Misinformation Be Detected?](https://mlsanity.com/paper/2309.13788/) (2023)
- [Fine-Grained Analysis of Propaganda in News Articles](https://mlsanity.com/paper/1910.02517/) (2019)
- [Winning Arguments: Interaction Dynamics and Persuasion Strategies in Good-faith Online Discussions](https://mlsanity.com/paper/1602.01103/) (2016)
- [SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles](https://mlsanity.com/paper/2009.02696/) (2020)
- [Fighting Fire with Fire: The Dual Role of LLMs in Crafting and Detecting Elusive Disinformation](https://mlsanity.com/paper/2310.15515/) (2023)
- [SemEval-2021 Task 6: Detection of Persuasion Techniques in Texts and Images](https://mlsanity.com/paper/2105.09284/) (2021)
- [Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4](https://mlsanity.com/paper/2305.14928/) (2023)
- [Artificial Influence: An Analysis Of AI-Driven Persuasion](https://mlsanity.com/paper/2303.08721/) (2023)
- [Measuring and Benchmarking Large Language Models' Capabilities to Generate Persuasive Language](https://mlsanity.com/paper/2406.17753/) (2024)
- [Better than Average: Paired Evaluation of NLP Systems](https://mlsanity.com/paper/2110.10746/) (2021)
- [StyloMetrix: An Open-Source Multilingual Tool for Representing Stylometric Vectors](https://mlsanity.com/paper/2309.12810/) (2023)
- [Changing Views: Persuasion Modeling and Argument Extraction from Online Discussions](https://mlsanity.com/paper/1907.06076/) (2019)
- [PCoT: Persuasion-Augmented Chain of Thought for Detecting Fake News and Social Media Disinformation](https://mlsanity.com/paper/2506.06842/) (2025)
- [Persuasive Natural Language Generation -- A Literature Review](https://mlsanity.com/paper/2101.05786/) (2021)
- [Disentangling Linguistic Features with Dimension-Wise Analysis of Vector Embeddings](https://mlsanity.com/paper/2504.14766/) (2025)

## Built on by (most cited first, 1 of 1)

- [Train Yourself as an LLM: Exploring Effects of AI Literacy on Persuasion via Role-playing LLM Training](https://mlsanity.com/paper/2604.02637/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) can generate highly persuasive text, raising concerns about their misuse for propaganda, manipulation, and other harmful purposes. This leads us to our central question: Is LLM-generated persuasion more difficult to automatically detect than human-written persuasion? To address this, we categorize controllable generation approaches for producing persuasive content with LLMs and introduce Persuaficial, a high-quality multilingual benchmark covering six languages: English, German, Polish, Italian, French and Russian. Using this benchmark, we conduct extensive empirical evaluations comparing human-authored and LLM-generated persuasive texts. We find that although overtly persuasive LLM-generated texts can be easier to detect than human-written ones, subtle LLM-generated persuasion consistently degrades automatic detection performance. Beyond detection performance, we provide the first comprehensive linguistic analysis contrasting human and LLM-generated persuasive texts, offering insights that may guide the development of more interpretable and robust detection tools.