# Generative Personas That Behave and Experience Like Humans

arXiv: https://arxiv.org/abs/2209.00459 · HTML: https://mlsanity.com/paper/2209.00459/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 47 references that are in the map.
- First posted on arXiv 26 Aug 2022.

## Builds on (most cited first, 22 of 22)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [First return, then explore](https://mlsanity.com/paper/2004.12919/) (2020)
- [PCGRL: Procedural Content Generation via Reinforcement Learning](https://mlsanity.com/paper/2001.09212/) (2020)
- [Learning Montezuma's Revenge from a Single Demonstration](https://mlsanity.com/paper/1812.03381/) (2018)
- [BeBold: Exploration Beyond the Boundary of Explored Regions](https://mlsanity.com/paper/2012.08621/) (2020)
- [Friend, Collaborator, Student, Manager: How Design of an AI-Driven Game Level Editor Affects Creators](https://mlsanity.com/paper/1901.06417/) (2019)
- [Automated Playtesting with Procedural Personas through MCTS with Evolved Heuristics](https://mlsanity.com/paper/1802.06881/) (2018)
- [Experience-Driven PCG via Reinforcement Learning: A Super Mario Bros Study](https://mlsanity.com/paper/2106.15877/) (2021)
- [Emotion in Reinforcement Learning Agents and Robots: A Survey](https://mlsanity.com/paper/1705.05172/) (2017)
- [Exploration Based Language Learning for Text-Based Games](https://mlsanity.com/paper/2001.08868/) (2020)
- [The Arousal video Game AnnotatIoN (AGAIN) Dataset](https://mlsanity.com/paper/2104.02643/) (2021)
- [Mixed-Initiative Level Design with RL Brush](https://mlsanity.com/paper/2008.02778/) (2020)
- [PAGAN: Video Affect Annotation Made Easy](https://mlsanity.com/paper/1907.01008/) (2019)
- [PBCS : Efficient Exploration and Exploitation Using a Synergy between Reinforcement Learning and Motion Planning](https://mlsanity.com/paper/2004.11667/) (2020)
- [How To Avoid Being Eaten By a Grue: Exploration Strategies for Text-Adventure Agents](https://mlsanity.com/paper/2002.08795/) (2020)
- [Adaptive Stress Testing without Domain Heuristics using Go-Explore](https://mlsanity.com/paper/2004.04292/) (2020)
- [Generating Diverse and Competitive Play-Styles for Strategy Games](https://mlsanity.com/paper/2104.08641/) (2021)
- [Go-Blend behavior and affect](https://mlsanity.com/paper/2109.13388/) (2021)
- [Deceptive Games](https://mlsanity.com/paper/1802.00048/) (2018)
- [Adapting Procedural Content Generation to Player Personas Through Evolution](https://mlsanity.com/paper/2112.04406/) (2021)

## Built on by (most cited first, 7 of 7)

- [Affective Game Computing: A Survey](https://mlsanity.com/paper/2309.14104/) (2023)
- [AI for Games in the Foundation Model Era](https://mlsanity.com/paper/2609.16679/) (2026)
- [Affectively Framework: Towards Human-like Affect-Based Agents](https://mlsanity.com/paper/2407.18316/) (2024)
- [Curiosity-driven Exploration in Sparse-reward Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2302.10825/) (2023)
- [Closing the Affective Loop via Experience-Driven Reinforcement Learning Designers](https://mlsanity.com/paper/2408.06346/) (2024)
- [Evolutionary Machine Learning and Games](https://mlsanity.com/paper/2311.16172/) (2023)
- [UBCL: A Reinforcement Learning Framework for Controllable and Diverse Player Behaviors](https://mlsanity.com/paper/2512.10835/) (2025)

## Abstract (arXiv)

Using artificial intelligence (AI) to automatically test a game remains a critical challenge for the development of richer and more complex game worlds and for the advancement of AI at large. One of the most promising methods for achieving that long-standing goal is the use of generative AI agents, namely procedural personas, that attempt to imitate particular playing behaviors which are represented as rules, rewards, or human demonstrations. All research efforts for building those generative agents, however, have focused solely on playing behavior which is arguably a narrow perspective of what a player actually does in a game. Motivated by this gap in the existing state of the art, in this paper we extend the notion of behavioral procedural personas to cater for player experience, thus examining generative agents that can both behave and experience their game as humans would. For that purpose, we employ the Go-Explore reinforcement learning paradigm for training human-like procedural personas, and we test our method on behavior and experience demonstrations of more than 100 players of a racing game. Our findings suggest that the generated agents exhibit distinctive play styles and experience responses of the human personas they were designed to imitate. Importantly, it also appears that experience, which is tied to playing behavior, can be a highly informative driver for better behavioral exploration.