# A Hybrid Theory and Data-driven Approach to Persuasion Detection with Large Language Models

arXiv: https://arxiv.org/abs/2511.22109 · HTML: https://mlsanity.com/paper/2511.22109/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 59 references that are in the map.
- First posted on arXiv 27 Nov 2025.

## Builds on (most cited first, 5 of 5)

- [Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good](https://mlsanity.com/paper/1906.06725/) (2019)
- [Large Language Models are Inconsistent and Biased Evaluators](https://mlsanity.com/paper/2405.01724/) (2024)
- [Argument Strength is in the Eye of the Beholder: Audience Effects in Persuasion](https://mlsanity.com/paper/1708.09085/) (2017)
- [Changing Views: Persuasion Modeling and Argument Extraction from Online Discussions](https://mlsanity.com/paper/1907.06076/) (2019)
- [Promoting and countering misinformation during Australia's 2019-2020 bushfires: A case study of polarisation](https://mlsanity.com/paper/2201.03153/) (2022)

## Built on by (most cited first, 2 of 2)

- [Detecting Winning Arguments with Large Language Models and Persuasion Strategies](https://mlsanity.com/paper/2601.10660/) (2026)
- [A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing](https://mlsanity.com/paper/2606.05330/) (2026)

## Abstract (arXiv)

Traditional psychological models of belief revision focus on face-to-face interactions, but with the rise of social media, more effective models are needed to capture belief revision at scale, in this rich text-based online discourse. Here, we use a hybrid approach, utilizing large language models (LLMs) to develop a model that predicts successful persuasion using features derived from psychological experiments. Our approach leverages LLM generated ratings of features previously examined in the literature to build a random forest classification model that predicts whether a message will result in belief change. Of the eight features tested, textitepistemic emotion and textitwillingness to share were the top-ranking predictors of belief change in the model. Our findings provide insights into the characteristics of persuasive messages and demonstrate how LLMs can enhance models of successful persuasion based on psychological theory. Given these insights, this work has broader applications in fields such as online influence detection and misinformation mitigation, as well as measuring the effectiveness of online narratives.