# Play with Emotion: Affect-Driven Reinforcement Learning

arXiv: https://arxiv.org/abs/2208.12622 · HTML: https://mlsanity.com/paper/2208.12622/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 26 Aug 2022.

## Builds on (most cited first, 16 of 16)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [First return, then explore](https://mlsanity.com/paper/2004.12919/) (2020)
- [BeBold: Exploration Beyond the Boundary of Explored Regions](https://mlsanity.com/paper/2012.08621/) (2020)
- [Emotion in Reinforcement Learning Agents and Robots: A Survey](https://mlsanity.com/paper/1705.05172/) (2017)
- [The Arousal video Game AnnotatIoN (AGAIN) Dataset](https://mlsanity.com/paper/2104.02643/) (2021)
- [Deep Structured Learning for Facial Action Unit Intensity Estimation](https://mlsanity.com/paper/1704.04481/) (2017)
- [The Pixels and Sounds of Emotion: General-Purpose Representations of Arousal in Games](https://mlsanity.com/paper/2101.10706/) (2021)
- [PAGAN: Video Affect Annotation Made Easy](https://mlsanity.com/paper/1907.01008/) (2019)
- [PBCS : Efficient Exploration and Exploitation Using a Synergy between Reinforcement Learning and Motion Planning](https://mlsanity.com/paper/2004.11667/) (2020)
- [How To Avoid Being Eaten By a Grue: Exploration Strategies for Text-Adventure Agents](https://mlsanity.com/paper/2002.08795/) (2020)
- [Show me the Way: Intrinsic Motivation from Demonstrations](https://mlsanity.com/paper/2006.12917/) (2020)
- [Go-Blend behavior and affect](https://mlsanity.com/paper/2109.13388/) (2021)
- [Deceptive Games](https://mlsanity.com/paper/1802.00048/) (2018)
- [Multi-Instance Dynamic Ordinal Random Fields for Weakly-supervised Facial Behavior Analysis](https://mlsanity.com/paper/1803.00907/) (2018)
- [AffRankNet+: Ranking Affect Using Privileged Information](https://mlsanity.com/paper/2108.05598/) (2021)

## Built on by (most cited first, 6 of 6)

- [Affective Game Computing: A Survey](https://mlsanity.com/paper/2309.14104/) (2023)
- [Affectively Framework: Towards Human-like Affect-Based Agents](https://mlsanity.com/paper/2407.18316/) (2024)
- [Curiosity-driven Exploration in Sparse-reward Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2302.10825/) (2023)
- [Expanding the Role of Affective Phenomena in Multimodal Interaction Research](https://mlsanity.com/paper/2305.10827/) (2023)
- [Closing the Affective Loop via Experience-Driven Reinforcement Learning Designers](https://mlsanity.com/paper/2408.06346/) (2024)
- [CosmoCore-Evo: Evolutionary Dream-Replay Reinforcement Learning for Adaptive Code Generation](https://mlsanity.com/paper/2512.21351/) (2025)

## Abstract (arXiv)

This paper introduces a paradigm shift by viewing the task of affect modeling as a reinforcement learning (RL) process. According to the proposed paradigm, RL agents learn a policy (i.e. affective interaction) by attempting to maximize a set of rewards (i.e. behavioral and affective patterns) via their experience with their environment (i.e. context). Our hypothesis is that RL is an effective paradigm for interweaving affect elicitation and manifestation with behavioral and affective demonstrations. Importantly, our second hypothesis-building on Damasio's somatic marker hypothesis-is that emotion can be the facilitator of decision-making. We test our hypotheses in a racing game by training Go-Blend agents to model human demonstrations of arousal and behavior; Go-Blend is a modified version of the Go-Explore algorithm which has recently showcased supreme performance in hard exploration tasks. We first vary the arousal-based reward function and observe agents that can effectively display a palette of affect and behavioral patterns according to the specified reward. Then we use arousal-based state selection mechanisms in order to bias the strategies that Go-Blend explores. Our findings suggest that Go-Blend not only is an efficient affect modeling paradigm but, more importantly, affect-driven RL improves exploration and yields higher performing agents, validating Damasio's hypothesis in the domain of games.