# How social reinforcement learning can lead to metastable polarisation and the voter model

arXiv: https://arxiv.org/abs/2406.07993 · HTML: https://mlsanity.com/paper/2406.07993/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 60 references that are in the map.
- First posted on arXiv 12 Jun 2024.

## Placed in

- Deep reinforcement learning methods / [RL training of language and agent models](https://mlsanity.com/deeprl/rl-training-of-language-and-agent-models/)

## Builds on (most cited first, 9 of 9)

- [The Spontaneous Emergence of Conventions: An Experimental Study of Cultural Evolution](https://mlsanity.com/paper/1502.06910/) (2015)
- [Deterministic limit of temporal difference reinforcement learning for stochastic games](https://mlsanity.com/paper/1809.07225/) (2018)
- [Opinion Polarization by Learning from Social Feedback](https://mlsanity.com/paper/1704.02890/) (2017)
- [The noisy voter model on complex networks](https://mlsanity.com/paper/1602.06935/) (2016)
- [Consensus Emerging from the Bottom-up: the Role of Cognitive Variables in Opinion Dynamics](https://mlsanity.com/paper/1502.06430/) (2015)
- [The dynamics of opinion expression](https://mlsanity.com/paper/1912.12631/) (2019)
- [Intrinsic fluctuations of reinforcement learning promote cooperation](https://mlsanity.com/paper/2209.01013/) (2022)
- [Biased processing and opinion polarization: experimental refinement of argument communication theory in the context of the energy debate](https://mlsanity.com/paper/2212.10117/) (2022)
- [Opinion dynamics beyond social influence](https://mlsanity.com/paper/2305.09398/) (2023)

## Abstract (arXiv)

Previous explanations for the persistence of polarization of opinions have typically included modelling assumptions that predispose the possibility of polarization (i.e., assumptions allowing a pair of agents to drift apart in their opinion such as repulsive interactions or bounded confidence). An exception is a recent simulation study showing that polarization is persistent when agents form their opinions using social reinforcement learning. Our goal is to highlight the usefulness of reinforcement learning in the context of modeling opinion dynamics, but that caution is required when selecting the tools used to study such a model. We show that the polarization observed in the model of the simulation study cannot persist indefinitely, and exhibits consensus asymptotically with probability one. By constructing a link between the reinforcement learning model and the voter model, we argue that the observed polarization is metastable. Finally, we show that a slight modification in the learning process of the agents changes the model from being non-ergodic to being ergodic. Our results show that reinforcement learning may be a powerful method for modelling polarization in opinion dynamics, but that the tools (objects to study such as the stationary distribution, or time to absorption for example) appropriate for analysing such models crucially depend on their properties (such as ergodicity, or transience). These properties are determined by the details of the learning process and may be difficult to identify based solely on simulations.