# Policy Distillation and Value Matching in Multiagent Reinforcement Learning

arXiv: https://arxiv.org/abs/1903.06592 · HTML: https://mlsanity.com/paper/1903.06592/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Mar 2019.

## Built on by (most cited first, 10 of 10)

- [Influencing Long-Term Behavior in Multiagent Reinforcement Learning](https://mlsanity.com/paper/2203.03535/) (2022)
- [Multi-Agent Policy Transfer via Task Relationship Modeling](https://mlsanity.com/paper/2203.04482/) (2022)
- [Explainable Action Advising for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2211.07882/) (2022)
- [KnowRU: Knowledge Reusing via Knowledge Distillation in Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2103.14891/) (2021)
- [Game-Theoretical Perspectives on Active Equilibria: A Preferred Solution Concept over Nash Equilibria](https://mlsanity.com/paper/2210.16175/) (2022)
- [Low-Rank Agent-Specific Adaptation (LoRASA) for Multi-Agent Policy Learning](https://mlsanity.com/paper/2502.05573/) (2025)
- [KnowSR: Knowledge Sharing among Homogeneous Agents in Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2105.11611/) (2021)
- [Multi-agent Policy Reciprocity with Theoretical Guarantee](https://mlsanity.com/paper/2304.05632/) (2023)
- [Natural Policy Gradient and Actor Critic Methods for Constrained Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2405.02456/) (2024)
- [Combining Trained Models in Reinforcement Learning](https://mlsanity.com/paper/2605.02159/) (2026)

## Abstract (arXiv)

Multiagent reinforcement learning algorithms (MARL) have been demonstrated on complex tasks that require the coordination of a team of multiple agents to complete. Existing works have focused on sharing information between agents via centralized critics to stabilize learning or through communication to increase performance, but do not generally look at how information can be shared between agents to address the curse of dimensionality in MARL. We posit that a multiagent problem can be decomposed into a multi-task problem where each agent explores a subset of the state space instead of exploring the entire state space. This paper introduces a multiagent actor-critic algorithm and method for combining knowledge from homogeneous agents through distillation and value-matching that outperforms policy distillation alone and allows further learning in both discrete and continuous action spaces.