# Closing the Affective Loop via Experience-Driven Reinforcement Learning Designers

arXiv: https://arxiv.org/abs/2408.06346 · HTML: https://mlsanity.com/paper/2408.06346/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods.
- Builds on 17 of its 54 references that are in the map.
- First posted on arXiv 23 Jul 2024.

## Placed in

- Deep reinforcement learning methods / [RL training of language and agent models](https://mlsanity.com/deeprl/rl-training-of-language-and-agent-models/)

## Builds on (most cited first, 17 of 17)

- [First return, then explore](https://mlsanity.com/paper/2004.12919/) (2020)
- [Unity: A General Platform for Intelligent Agents](https://mlsanity.com/paper/1809.02627/) (2018)
- [Large Language Models and Games: A Survey and Roadmap](https://mlsanity.com/paper/2402.18659/) (2024)
- [PCGRL: Procedural Content Generation via Reinforcement Learning](https://mlsanity.com/paper/2001.09212/) (2020)
- [Deep Learning for Procedural Content Generation](https://mlsanity.com/paper/2010.04548/) (2020)
- [Affect-Driven Dialog Generation](https://mlsanity.com/paper/1904.02793/) (2019)
- [Cascade EF-GAN: Progressive Facial Expression Editing with Local Focuses](https://mlsanity.com/paper/2003.05905/) (2020)
- [Experience-Driven PCG via Reinforcement Learning: A Super Mario Bros Study](https://mlsanity.com/paper/2106.15877/) (2021)
- [Fine-grained Affective Processing Capabilities Emerging from Large Language Models](https://mlsanity.com/paper/2309.01664/) (2023)
- [The Arousal video Game AnnotatIoN (AGAIN) Dataset](https://mlsanity.com/paper/2104.02643/) (2021)
- [The Pixels and Sounds of Emotion: General-Purpose Representations of Arousal in Games](https://mlsanity.com/paper/2101.10706/) (2021)
- [PAGAN: Video Affect Annotation Made Easy](https://mlsanity.com/paper/1907.01008/) (2019)
- [Affective Game Computing: A Survey](https://mlsanity.com/paper/2309.14104/) (2023)
- [Exploring ChatGPT's Empathic Abilities](https://mlsanity.com/paper/2308.03527/) (2023)
- [Generative Personas That Behave and Experience Like Humans](https://mlsanity.com/paper/2209.00459/) (2022)
- [Play with Emotion: Affect-Driven Reinforcement Learning](https://mlsanity.com/paper/2208.12622/) (2022)
- [A Review of Affective Generation Models](https://mlsanity.com/paper/2202.10763/) (2022)

## Built on by (most cited first, 1 of 1)

- [Recursive Game Creator: An Agentic Product-Level Experience-Oriented Game Harness](https://mlsanity.com/paper/2610.08621/) (2026)

## Abstract (arXiv)

Autonomously tailoring content to a set of predetermined affective patterns has long been considered the holy grail of affect-aware human-computer interaction at large. The experience-driven procedural content generation framework realises this vision by searching for content that elicits a certain experience pattern to a user. In this paper, we propose a novel reinforcement learning (RL) framework for generating affect-tailored content, and we test it in the domain of racing games. Specifically, the experience-driven RL (EDRL) framework is given a target arousal trace, and it then generates a racetrack that elicits the desired affective responses for a particular type of player. EDRL leverages a reward function that assesses the affective pattern of any generated racetrack from a corpus of arousal traces. Our findings suggest that EDRL can accurately generate affect-driven racing game levels according to a designer's style and outperforms search-based methods for personalised content generation. The method is not only directly applicable to game content generation tasks but also employable broadly to any domain that uses content for affective adaptation.