# Exploring the Impact of Tunable Agents in Sequential Social Dilemmas

arXiv: https://arxiv.org/abs/2101.11967 · HTML: https://mlsanity.com/paper/2101.11967/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 26 references that are in the map.
- First posted on arXiv 28 Jan 2021.

## Builds on (most cited first, 6 of 6)

- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [A Survey of Multi-Objective Sequential Decision-Making](https://mlsanity.com/paper/1402.0590/) (2014)
- [Multi-agent Reinforcement Learning in Sequential Social Dilemmas](https://mlsanity.com/paper/1702.03037/) (2017)
- [Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Survey](https://mlsanity.com/paper/1909.02964/) (2019)
- [A utility-based analysis of equilibria in multi-objective normal form games](https://mlsanity.com/paper/2001.08177/) (2020)
- [Opponent Learning Awareness and Modelling in Multi-Objective Normal Form Games](https://mlsanity.com/paper/2011.07290/) (2020)

## Built on by (most cited first, 2 of 2)

- [Multi-Agent, Human-Agent and Beyond: A Survey on Cooperation in Social Dilemmas](https://mlsanity.com/paper/2402.17270/) (2024)
- [Expected Scalarised Returns Dominance: A New Solution Concept for Multi-Objective Decision Making](https://mlsanity.com/paper/2106.01048/) (2021)

## Abstract (arXiv)

When developing reinforcement learning agents, the standard approach is to train an agent to converge to a fixed policy that is as close to optimal as possible for a single fixed reward function. If different agent behaviour is required in the future, an agent trained in this way must normally be either fully or partially retrained, wasting valuable time and resources. In this study, we leverage multi-objective reinforcement learning to create tunable agents, i.e. agents that can adopt a range of different behaviours according to the designer's preferences, without the need for retraining. We apply this technique to sequential social dilemmas, settings where there is inherent tension between individual and collective rationality. Learning a single fixed policy in such settings leaves one at a significant disadvantage if the opponents' strategies change after learning is complete. In our work, we demonstrate empirically that the tunable agents framework allows easy adaption between cooperative and competitive behaviours in sequential social dilemmas without the need for retraining, allowing a single trained agent model to be adjusted to cater for a wide range of behaviours and opponent strategies.