# Inverse Attention Agents for Multi-Agent Systems

arXiv: https://arxiv.org/abs/2410.21794 · HTML: https://mlsanity.com/paper/2410.21794/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 30 references that are in the map.
- First posted on arXiv 29 Oct 2024.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Emergence of Grounded Compositional Language in Multi-Agent Populations](https://mlsanity.com/paper/1703.04908/) (2017)
- [Machine Theory of Mind](https://mlsanity.com/paper/1802.07740/) (2018)
- [Learning Latent Representations to Influence Multi-Agent Interaction](https://mlsanity.com/paper/2011.06619/) (2020)
- [ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mind](https://mlsanity.com/paper/2111.09189/) (2021)
- [Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2003.10423/) (2020)
- [Towards Open Ad Hoc Teamwork Using Graph-based Policy Learning](https://mlsanity.com/paper/2006.10412/) (2020)
- [Theory of Minds: Understanding Behavior in Groups Through Inverse Planning](https://mlsanity.com/paper/1901.06085/) (2019)
- [Deep Interactive Bayesian Reinforcement Learning via Meta-Learning](https://mlsanity.com/paper/2101.03864/) (2021)
- [Reasoning about Hypothetical Agent Behaviours and their Parameters](https://mlsanity.com/paper/1906.11064/) (2019)
- [Expected Value of Communication for Planning in Ad Hoc Teamwork](https://mlsanity.com/paper/2103.01171/) (2021)
- [Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks](https://mlsanity.com/paper/2007.12803/) (2020)
- [Modeling Communication to Coordinate Perspectives in Cooperation](https://mlsanity.com/paper/2106.02164/) (2021)
- [SocialGFs: Learning Social Gradient Fields for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2405.01839/) (2024)

## Built on by (most cited first, 3 of 3)

- [TeamCraft: A Benchmark for Multi-Modal Multi-Agent Systems in Minecraft](https://mlsanity.com/paper/2412.05255/) (2024)
- [Partner Modelling Emerges in Recurrent Agents (But Only When It Matters)](https://mlsanity.com/paper/2505.17323/) (2025)
- [Topology Enhanced MARL for Multi-Agent Cooperative Decision-Making of CAVs](https://mlsanity.com/paper/2507.12110/) (2025)

## Abstract (arXiv)

A major challenge for Multi-Agent Systems is enabling agents to adapt dynamically to diverse environments in which opponents and teammates may continually change. Agents trained using conventional methods tend to excel only within the confines of their training cohorts; their performance drops significantly when confronting unfamiliar agents. To address this shortcoming, we introduce Inverse Attention Agents that adopt concepts from the Theory of Mind (ToM) implemented algorithmically using an attention mechanism trained in an end-to-end manner. Crucial to determining the final actions of these agents, the weights in their attention model explicitly represent attention to different goals. We furthermore propose an inverse attention network that deduces the ToM of agents based on observations and prior actions. The network infers the attentional states of other agents, thereby refining the attention weights to adjust the agent's final action. We conduct experiments in a continuous environment, tackling demanding tasks encompassing cooperation, competition, and a blend of both. They demonstrate that the inverse attention network successfully infers the attention of other agents, and that this information improves agent performance. Additional human experiments show that, compared to baseline agent models, our inverse attention agents exhibit superior cooperation with humans and better emulate human behaviors.