# Emotion in Reinforcement Learning Agents and Robots: A Survey

arXiv: https://arxiv.org/abs/1705.05172 · HTML: https://mlsanity.com/paper/1705.05172/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 May 2017.

## Built on by (most cited first, 21 of 21)

- [Levels of explainable artificial intelligence for human-aligned conversational explanations](https://mlsanity.com/paper/2107.03178/) (2021)
- [Generative Personas That Behave and Experience Like Humans](https://mlsanity.com/paper/2209.00459/) (2022)
- [Go-Blend behavior and affect](https://mlsanity.com/paper/2109.13388/) (2021)
- [Play with Emotion: Affect-Driven Reinforcement Learning](https://mlsanity.com/paper/2208.12622/) (2022)
- [Adaptive Rollout Length for Model-Based RL Using Model-Free Deep RL](https://mlsanity.com/paper/2206.02380/) (2022)
- [Survey and Perspective on Social Emotions in Robotics](https://mlsanity.com/paper/2105.09647/) (2021)
- [Affectively Framework: Towards Human-like Affect-Based Agents](https://mlsanity.com/paper/2407.18316/) (2024)
- [Noe: Norms Emergence and Robustness Based on Emotions in Multiagent Systems](https://mlsanity.com/paper/2104.15034/) (2021)
- [Life-inspired Interoceptive Artificial Intelligence for Autonomous and Adaptive Agents](https://mlsanity.com/paper/2309.05999/) (2023)
- [Intuitions of Compromise: Utilitarianism vs. Contractualism](https://mlsanity.com/paper/2410.05496/) (2024)
- [Ailed: A Psyche-Driven Chess Engine with Dynamic Emotional Modulation](https://mlsanity.com/paper/2603.05352/) (2026)
- [Emotions as abstract evaluation criteria in biological and artificial intelligences](https://mlsanity.com/paper/2111.15275/) (2021)
- [Incorporating Rivalry in Reinforcement Learning for a Competitive Game](https://mlsanity.com/paper/2208.10327/) (2022)
- [Towards Understanding Human Emotional Fluctuations with Sparse Check-In Data](https://mlsanity.com/paper/2409.06863/) (2024)
- [Emotion-Gradient Metacognitive RSI (Part I): Theoretical Foundations and Single-Agent Architecture](https://mlsanity.com/paper/2505.07757/) (2025)
- [The Emotion-Memory Link: Do Memorability Annotations Matter for Intelligent Systems?](https://mlsanity.com/paper/2507.14084/) (2025)
- [CosmoCore-Evo: Evolutionary Dream-Replay Reinforcement Learning for Adaptive Code Generation](https://mlsanity.com/paper/2512.21351/) (2025)
- [Emotional regulation improves deep learning-based image classification](https://mlsanity.com/paper/2606.13081/) (2026)
- [A Transdiagnostic Space of Disorder Like Phenotypes in Reinforcement Learning Agents](https://mlsanity.com/paper/2607.07753/) (2026)
- [Regret Dominates Surprise: Design-Time Requirements Engineering for Agentic-AI Safety](https://mlsanity.com/paper/2609.05692/) (2026)
- [Functional Emotion Without Character: Large Language Models, Aristotelian Disposition, and the Limits of Behavioral Alignment](https://mlsanity.com/paper/2609.22362/) (2026)

## Abstract (arXiv)

This article provides the first survey of computational models of emotion in reinforcement learning (RL) agents. The survey focuses on agent/robot emotions, and mostly ignores human user emotions. Emotions are recognized as functional in decision-making by influencing motivation and action selection. Therefore, computational emotion models are usually grounded in the agent's decision making architecture, of which RL is an important subclass. Studying emotions in RL-based agents is useful for three research fields. For machine learning (ML) researchers, emotion models may improve learning efficiency. For the interactive ML and human-robot interaction (HRI) community, emotions can communicate state and enhance user investment. Lastly, it allows affective modelling (AM) researchers to investigate their emotion theories in a successful AI agent class. This survey provides background on emotion theory and RL. It systematically addresses 1) from what underlying dimensions (e.g., homeostasis, appraisal) emotions can be derived and how these can be modelled in RL-agents, 2) what types of emotions have been derived from these dimensions, and 3) how these emotions may either influence the learning efficiency of the agent or be useful as social signals. We also systematically compare evaluation criteria, and draw connections to important RL sub-domains like (intrinsic) motivation and model-based RL. In short, this survey provides both a practical overview for engineers wanting to implement emotions in their RL agents, and identifies challenges and directions for future emotion-RL research.