# Contextual Knowledge Sharing in Multi-Agent Reinforcement Learning with Decentralized Communication and Coordination

arXiv: https://arxiv.org/abs/2501.15695 · HTML: https://mlsanity.com/paper/2501.15695/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 48 references that are in the map.
- First posted on arXiv 26 Jan 2025.

## Builds on (most cited first, 26 of 26)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2003.08839/) (2020)
- [Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?](https://mlsanity.com/paper/2011.09533/) (2020)
- [Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps](https://mlsanity.com/paper/2209.12836/) (2022)
- [TarMAC: Targeted Multi-Agent Communication](https://mlsanity.com/paper/1810.11187/) (2018)
- [Learning Attentional Communication for Multi-Agent Cooperation](https://mlsanity.com/paper/1805.07733/) (2018)
- [When2com: Multi-Agent Perception via Communication Graph Grouping](https://mlsanity.com/paper/2006.00176/) (2020)
- [ROMA: Multi-Agent Reinforcement Learning with Emergent Roles](https://mlsanity.com/paper/2003.08039/) (2020)
- [Who2com: Collaborative Perception via Learnable Handshake Communication](https://mlsanity.com/paper/2003.09575/) (2020)
- [Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks](https://mlsanity.com/paper/1812.09755/) (2018)
- [Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2006.07169/) (2020)
- [V-Learning -- A Simple, Efficient, Decentralized Algorithm for Multiagent RL](https://mlsanity.com/paper/2110.14555/) (2021)
- [Deep Coordination Graphs](https://mlsanity.com/paper/1910.00091/) (2019)
- [Shapley Q-value: A Local Reward Approach to Solve Global Reward Games](https://mlsanity.com/paper/1907.05707/) (2019)
- [Learning Individually Inferred Communication for Multi-Agent Cooperation](https://mlsanity.com/paper/2006.06455/) (2020)
- [The Complexity of Markov Equilibrium in Stochastic Games](https://mlsanity.com/paper/2204.03991/) (2022)
- [Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Control](https://mlsanity.com/paper/1909.02682/) (2019)
- [Scalable Multi-Agent Reinforcement Learning through Intelligent Information Aggregation](https://mlsanity.com/paper/2211.02127/) (2022)
- [Learn to Follow: Decentralized Lifelong Multi-agent Pathfinding via Planning and Learning](https://mlsanity.com/paper/2310.01207/) (2023)
- [A Survey on Context-Aware Multi-Agent Systems: Techniques, Challenges and Future Directions](https://mlsanity.com/paper/2402.01968/) (2024)
- [GCS: Graph-based Coordination Strategy for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2201.06257/) (2022)
- [Learning Multi-Agent Coordination through Connectivity-driven Communication](https://mlsanity.com/paper/2002.05233/) (2020)
- [Settling Decentralized Multi-Agent Coordinated Exploration by Novelty Sharing](https://mlsanity.com/paper/2402.02097/) (2024)
- [Cautiously-Optimistic Knowledge Sharing for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2312.12095/) (2023)

## Built on by (most cited first, 2 of 2)

- [Intelligent System of Emergent Knowledge: A Coordination Fabric for Billions of Minds](https://mlsanity.com/paper/2506.09335/) (2025)
- [When Does Hierarchy Help? Benchmarking Agent Coordination in Event-Driven Industrial Scheduling](https://mlsanity.com/paper/2605.13172/) (2026)

## Abstract (arXiv)

Decentralized Multi-Agent Reinforcement Learning (Dec-MARL) has emerged as a pivotal approach for addressing complex tasks in dynamic environments. Existing Multi-Agent Reinforcement Learning (MARL) methodologies typically assume a shared objective among agents and rely on centralized control. However, many real-world scenarios feature agents with individual goals and limited observability of other agents, complicating coordination and hindering adaptability. Existing Dec-MARL strategies prioritize either communication or coordination, lacking an integrated approach that leverages both. This paper presents a novel Dec-MARL framework that integrates peer-to-peer communication and coordination, incorporating goal-awareness and time-awareness into the agents' knowledge-sharing processes. Our framework equips agents with the ability to (i) share contextually relevant knowledge to assist other agents, and (ii) reason based on information acquired from multiple agents, while considering their own goals and the temporal context of prior knowledge. We evaluate our approach through several complex multi-agent tasks in environments with dynamically appearing obstacles. Our work demonstrates that incorporating goal-aware and time-aware knowledge sharing significantly enhances overall performance.