# Learning to Communicate Implicitly By Actions

arXiv: https://arxiv.org/abs/1810.04444 · HTML: https://mlsanity.com/paper/1810.04444/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2018.

## Built on by (most cited first, 11 of 11)

- [Model-based Multi-agent Policy Optimization with Adaptive Opponent-wise Rollouts](https://mlsanity.com/paper/2105.03363/) (2021)
- [Differentiable Multi-Agent Actor-Critic for Multi-Step Radiology Report Summarization](https://mlsanity.com/paper/2203.08257/) (2022)
- [From Explicit Communication to Tacit Cooperation:A Novel Paradigm for Cooperative MARL](https://mlsanity.com/paper/2304.14656/) (2023)
- [Generalized Beliefs for Cooperative AI](https://mlsanity.com/paper/2206.12765/) (2022)
- [Multi-agent Actor-Critic with Time Dynamical Opponent Model](https://mlsanity.com/paper/2204.05576/) (2022)
- [Investigating the Development of Task-Oriented Communication in Vision-Language Models](https://mlsanity.com/paper/2601.20641/) (2026)
- [Structured Diversification Emergence via Reinforced Organization Control and Hierarchical Consensus Learning](https://mlsanity.com/paper/2102.04775/) (2021)
- [CtD: Composition through Decomposition in Emergent Communication](https://mlsanity.com/paper/2601.10169/) (2026)
- [Learning to Infer Belief Embedded Communication](https://mlsanity.com/paper/2203.07832/) (2022)
- [BridgeHand2Vec Bridge Hand Representation](https://mlsanity.com/paper/2310.06624/) (2023)
- [MARLEM: A Multi-Agent Reinforcement Learning Simulation Framework for Implicit Cooperation in Decentralized Local Energy Markets](https://mlsanity.com/paper/2602.16063/) (2026)

## Abstract (arXiv)

In situations where explicit communication is limited, human collaborators act by learning to: (i) infer meaning behind their partner's actions, and (ii) convey private information about the state to their partner implicitly through actions. The first component of this learning process has been well-studied in multi-agent systems, whereas the second --- which is equally crucial for successful collaboration --- has not. To mimic both components mentioned above, thereby completing the learning process, we introduce a novel algorithm: Policy Belief Learning (PBL). PBL uses a belief module to model the other agent's private information and a policy module to form a distribution over actions informed by the belief module. Furthermore, to encourage communication by actions, we propose a novel auxiliary reward which incentivizes one agent to help its partner to make correct inferences about its private information. The auxiliary reward for communication is integrated into the learning of the policy module. We evaluate our approach on a set of environments including a matrix game, particle environment and the non-competitive bidding problem from contract bridge. We show empirically that this auxiliary reward is effective and easy to generalize. These results demonstrate that our PBL algorithm can produce strong pairs of agents in collaborative games where explicit communication is disabled.