# Reliably Re-Acting to Partner's Actions with the Social Intrinsic Motivation of Transfer Empowerment

arXiv: https://arxiv.org/abs/2203.03355 · HTML: https://mlsanity.com/paper/2203.03355/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 7 Mar 2022.

## Builds on (most cited first, 17 of 17)

- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [Large-Scale Study of Curiosity-Driven Learning](https://mlsanity.com/paper/1808.04355/) (2018)
- [Variational Intrinsic Control](https://mlsanity.com/paper/1611.07507/) (2016)
- ["Other-Play" for Zero-Shot Coordination](https://mlsanity.com/paper/2003.02979/) (2020)
- [MAVEN: Multi-Agent Variational Exploration](https://mlsanity.com/paper/1910.07483/) (2019)
- [Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data](https://mlsanity.com/paper/1605.06432/) (2016)
- [Influence-Based Multi-Agent Exploration](https://mlsanity.com/paper/1910.05512/) (2019)
- [Unsupervised Real-Time Control through Variational Empowerment](https://mlsanity.com/paper/1710.05101/) (2017)
- [Learning to Share and Hide Intentions using Information Regularization](https://mlsanity.com/paper/1808.02093/) (2018)
- [Adversarially Guided Self-Play for Adopting Social Conventions](https://mlsanity.com/paper/2001.05994/) (2020)
- [A unified strategy for implementing curiosity and empowerment driven reinforcement learning](https://mlsanity.com/paper/1806.06505/) (2018)
- [Measuring collaborative emergent behavior in multi-agent reinforcement learning](https://mlsanity.com/paper/1807.08663/) (2018)
- [Social Navigation with Human Empowerment driven Deep Reinforcement Learning](https://mlsanity.com/paper/2003.08158/) (2020)
- [New And Surprising Ways to Be Mean. Adversarial NPCs with Coupled Empowerment Minimisation](https://mlsanity.com/paper/1806.01387/) (2018)

## Abstract (arXiv)

We consider multi-agent reinforcement learning (MARL) for cooperative communication and coordination tasks. MARL agents can be brittle because they can overfit their training partners' policies. This overfitting can produce agents that adopt policies that act under the expectation that other agents will act in a certain way rather than react to their actions. Our objective is to bias the learning process towards finding reactive strategies towards other agents' behaviors. Our method, transfer empowerment, measures the potential influence between agents' actions. Results from three simulated cooperation scenarios support our hypothesis that transfer empowerment improves MARL performance. We discuss how transfer empowerment could be a useful principle to guide multi-agent coordination by ensuring reactiveness to one's partner.