# Preference Communication in Multi-Objective Normal-Form Games

arXiv: https://arxiv.org/abs/2111.09191 · HTML: https://mlsanity.com/paper/2111.09191/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 44 references that are in the map.
- First posted on arXiv 17 Nov 2021.

## Builds on (most cited first, 15 of 15)

- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [A Survey of Multi-Objective Sequential Decision-Making](https://mlsanity.com/paper/1402.0590/) (2014)
- [Reinforcement Learning for Combinatorial Optimization: A Survey](https://mlsanity.com/paper/2003.03600/) (2020)
- [Optimization of Molecules via Deep Reinforcement Learning](https://mlsanity.com/paper/1810.08678/) (2018)
- [Multiagent Cooperation and Competition with Deep Reinforcement Learning](https://mlsanity.com/paper/1511.08779/) (2015)
- [Multi-Objective Deep Reinforcement Learning](https://mlsanity.com/paper/1610.02707/) (2016)
- [Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Survey](https://mlsanity.com/paper/1909.02964/) (2019)
- [Learning to Play No-Press Diplomacy with Best Response Policy Iteration](https://mlsanity.com/paper/2006.04635/) (2020)
- [Nash Convergence of Gradient Dynamics in Iterated General-Sum Games](https://mlsanity.com/paper/1301.3892/) (2013)
- [Ordered Preference Elicitation Strategies for Supporting Multi-Objective Decision Making](https://mlsanity.com/paper/1802.07606/) (2018)
- [Emergent Communication under Competition](https://mlsanity.com/paper/2101.10276/) (2021)
- [A utility-based analysis of equilibria in multi-objective normal form games](https://mlsanity.com/paper/2001.08177/) (2020)
- [Opponent Learning Awareness and Modelling in Multi-Objective Normal Form Games](https://mlsanity.com/paper/2011.07290/) (2020)
- [Multi-objective Stackelberg Game Between a Regulating Authority and a Mining Company: A Case Study in Environmental Economics](https://mlsanity.com/paper/1307.6250/) (2013)
- [On Existence, Mixtures, Computation and Efficiency in Multi-objective Games](https://mlsanity.com/paper/1809.04979/) (2018)

## Built on by (most cited first, 3 of 3)

- [Emergent Language: A Survey and Taxonomy](https://mlsanity.com/paper/2409.02645/) (2024)
- [MOMAland: A Set of Benchmarks for Multi-Objective Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2407.16312/) (2024)
- [Learning in Repeated Multi-Objective Stackelberg Games with Payoff Manipulation](https://mlsanity.com/paper/2508.14705/) (2025)

## Abstract (arXiv)

We consider preference communication in two-player multi-objective normal-form games. In such games, the payoffs resulting from joint actions are vector-valued. Taking a utility-based approach, we assume there exists a utility function for each player which maps vectors to scalar utilities and consider agents that aim to maximise the utility of expected payoff vectors. As agents typically do not know their opponent's utility function or strategy, they must learn policies to interact with each other. Inspired by Stackelberg games, we introduce four novel preference communication protocols to aid agents in arriving at adequate solutions. Each protocol describes a specific approach for one agent to communicate preferences over their actions and how another agent responds. Additionally, to study when communication emerges, we introduce a communication protocol where agents must learn when to communicate. These protocols are subsequently evaluated on a set of five benchmark games against baseline agents that do not communicate. We find that preference communication can alter the learning process and lead to the emergence of cyclic policies which had not been previously observed in this setting. We further observe that the resulting policies can heavily depend on the characteristics of the game that is played. Lastly, we find that communication naturally emerges in both cooperative and self-interested settings.