# A Meta-Analysis of the Persuasive Power of Large Language Models

arXiv: https://arxiv.org/abs/2512.01431 · HTML: https://mlsanity.com/paper/2512.01431/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2025.

## Built on by (most cited first, 7 of 7)

- [Understanding Persuasive Interactions between Generative Social Agents and Humans: The Knowledge-based Persuasion Model (KPM)](https://mlsanity.com/paper/2602.11483/) (2026)
- [The Hidden Puppet Master: Predicting Human Belief Change in Manipulative LLM Dialogues](https://mlsanity.com/paper/2603.20907/) (2026)
- [AutoSynthesis: An agentic system for automated meta-analysis](https://mlsanity.com/paper/2607.15247/) (2026)
- [Never say never: Exploring the effects of available knowledge on agent persuasiveness in controlled physiotherapy motivation dialogues](https://mlsanity.com/paper/2602.12924/) (2026)
- [Human Decision-Making with Persuasive and Narrative LLM Explanations](https://mlsanity.com/paper/2605.23867/) (2026)
- [A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing](https://mlsanity.com/paper/2606.05330/) (2026)
- [Annie, Are You Okay? How Style- and Context-Based Personalization Shape AI-Assisted Decision-Making](https://mlsanity.com/paper/2609.24644/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly used for persuasion, such as in political communication and marketing, where they affect how people think, choose, and act. Yet, empirical findings on the effectiveness of LLMs in persuasion compared to humans remain inconsistent. The aim of this study was to systematically review and meta-analytically assess whether LLMs differ from humans in persuasive effectiveness. We identified 7 studies with 17,422 participants primarily recruited from English-speaking countries and 12 effect size estimates. Egger's test indicated potential small-study effects (p = .018), but the trim-and-fill analysis did not impute any missing studies, suggesting a low risk of publication bias. We then compute the standardized effect sizes based on Hedges' g. The results show no significant overall difference in persuasive performance between LLMs and humans (g = 0.02, p = .530). However, we observe substantial heterogeneity across studies (I2 = 75.97%), suggesting that persuasiveness strongly depends on contextual factors. In separate exploratory moderator analyses, no individual factor (e.g., LLM model, conversation design, or domain) reached statistical significance, which may be due to the limited number of studies. When considered jointly in a combined model, these factors explained a large proportion of the between-study variance (R2 = 81.93%), and residual heterogeneity is low (I2 = 35.51%). Although based on a small number of studies, this suggests that differences in LLM model, conversation design, and domain are important contextual factors in shaping persuasive performance, and that single-factor tests may understate their influence. Our results highlight that LLMs can match human performance in persuasion, but their success depends strongly on how they are implemented and embedded in communication contexts.