# Go-Blend behavior and affect

arXiv: https://arxiv.org/abs/2109.13388 · HTML: https://mlsanity.com/paper/2109.13388/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 38 references that are in the map.
- First posted on arXiv 24 Sep 2021.

## Builds on (most cited first, 15 of 15)

- [First return, then explore](https://mlsanity.com/paper/2004.12919/) (2020)
- [Go-Explore: a New Approach for Hard-Exploration Problems](https://mlsanity.com/paper/1901.10995/) (2019)
- [PCGRL: Procedural Content Generation via Reinforcement Learning](https://mlsanity.com/paper/2001.09212/) (2020)
- [Learning Montezuma's Revenge from a Single Demonstration](https://mlsanity.com/paper/1812.03381/) (2018)
- [Experience-Driven PCG via Reinforcement Learning: A Super Mario Bros Study](https://mlsanity.com/paper/2106.15877/) (2021)
- [Emotion in Reinforcement Learning Agents and Robots: A Survey](https://mlsanity.com/paper/1705.05172/) (2017)
- [Exploration Based Language Learning for Text-Based Games](https://mlsanity.com/paper/2001.08868/) (2020)
- [Deep Structured Learning for Facial Action Unit Intensity Estimation](https://mlsanity.com/paper/1704.04481/) (2017)
- [The Pixels and Sounds of Emotion: General-Purpose Representations of Arousal in Games](https://mlsanity.com/paper/2101.10706/) (2021)
- [From Pixels to Affect: A Study on Games and Player Experience](https://mlsanity.com/paper/1907.02288/) (2019)
- [PBCS : Efficient Exploration and Exploitation Using a Synergy between Reinforcement Learning and Motion Planning](https://mlsanity.com/paper/2004.11667/) (2020)
- [How To Avoid Being Eaten By a Grue: Exploration Strategies for Text-Adventure Agents](https://mlsanity.com/paper/2002.08795/) (2020)
- [Show me the Way: Intrinsic Motivation from Demonstrations](https://mlsanity.com/paper/2006.12917/) (2020)
- [Adaptive Stress Testing without Domain Heuristics using Go-Explore](https://mlsanity.com/paper/2004.04292/) (2020)
- [Multi-Instance Dynamic Ordinal Random Fields for Weakly-supervised Facial Behavior Analysis](https://mlsanity.com/paper/1803.00907/) (2018)

## Built on by (most cited first, 6 of 6)

- [Affective Game Computing: A Survey](https://mlsanity.com/paper/2309.14104/) (2023)
- [Generative Personas That Behave and Experience Like Humans](https://mlsanity.com/paper/2209.00459/) (2022)
- [Learning Task-Independent Game State Representations from Unlabeled Images](https://mlsanity.com/paper/2206.06490/) (2022)
- [Play with Emotion: Affect-Driven Reinforcement Learning](https://mlsanity.com/paper/2208.12622/) (2022)
- [Towards General Game Representations: Decomposing Games Pixels into Content and Style](https://mlsanity.com/paper/2307.11141/) (2023)
- [Game State Learning via Game Scene Augmentation](https://mlsanity.com/paper/2207.01289/) (2022)

## Abstract (arXiv)

This paper proposes a paradigm shift for affective computing by viewing the affect modeling task as a reinforcement learning process. According to our proposed framework the context (environment) and the actions of an agent define the common representation that interweaves behavior and affect. To realise this framework we build on recent advances in reinforcement learning and use a modified version of the Go-Explore algorithm which has showcased supreme performance in hard exploration tasks. In this initial study, we test our framework in an arcade game by training Go-Explore agents to both play optimally and attempt to mimic human demonstrations of arousal. We vary the degree of importance between optimal play and arousal imitation and create agents that can effectively display a palette of affect and behavioral patterns. Our Go-Explore implementation not only introduces a new paradigm for affect modeling; it empowers believable AI-based game testing by providing agents that can blend and express a multitude of behavioral and affective patterns.