# Evaluating the Capabilities of LLMs for Persuasive Dialogue

arXiv: https://arxiv.org/abs/2608.29738 · HTML: https://mlsanity.com/paper/2608.29738/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 46 references that are in the map.
- First posted on arXiv 30 Aug 2026.

## Builds on (most cited first, 7 of 7)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Winning Arguments: Interaction Dynamics and Persuasion Strategies in Good-faith Online Discussions](https://mlsanity.com/paper/1602.01103/) (2016)
- [The Persuasive Power of Large Language Models](https://mlsanity.com/paper/2312.15523/) (2023)
- [Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of LLM arguments](https://mlsanity.com/paper/2404.09329/) (2024)
- [The Thin Line Between Comprehension and Persuasion in LLMs](https://mlsanity.com/paper/2507.01936/) (2025)
- [Validating Political Position Predictions of Arguments](https://mlsanity.com/paper/2602.18351/) (2026)

## Abstract (arXiv)

Large language models (LLMs) can generate apparently highly persuasive text, but does sounding persuasive mean arguing well? We introduce textscPersuasio, a multi-agent dialogue platform grounded in a formal argumentation-based theory of persuasion dialogues that adjudicates logical winners during free-text debates. Using this system, we generated 192 debates on a UK political topic between humans and LLMs, and evaluated 22 interlocutors through both automated adjudication and 9,702 crowdsourced pairwise judgements across 1,386 annotation instances. We observed a consistent decoupling between subjective and formal persuasiveness: LLMs dominated the subjective ranking yet performed substantially worse under argumentation-theoretic adjudication, where humans remained competitive. Multi-agent and retrieval-augmented variants further widened this divergence. These findings reveal a systematic gap between rhetorical fluency and formal argumentative strength in LLM-based persuasive dialogues.