# Wasserstein-Barycenter Consensus for Cooperative Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2506.12497 · HTML: https://mlsanity.com/paper/2506.12497/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2025.

## Built on by (most cited first, 2 of 2)

- [Can Optimal Transport Improve Federated Inverse Reinforcement Learning?](https://mlsanity.com/paper/2601.00309/) (2026)
- [Federated Distributional Reinforcement Learning with Distributional Critic Regularization](https://mlsanity.com/paper/2603.17820/) (2026)

## Abstract (arXiv)

Cooperative multi-agent reinforcement learning (MARL) demands principled mechanisms to align heterogeneous policies while preserving the capacity for specialized behavior. We introduce a novel consensus framework that defines the team strategy as the entropic-regularized p-Wasserstein barycenter of agents' joint state--action visitation measures. By augmenting each agent's policy objective with a soft penalty proportional to its Sinkhorn divergence from this barycenter, the proposed approach encourages coherent group behavior without enforcing rigid parameter sharing. We derive an algorithm that alternates between Sinkhorn-barycenter computation and policy-gradient updates, and we prove that, under standard Lipschitz and compactness assumptions, the maximal pairwise policy discrepancy contracts at a geometric rate. Empirical evaluation on a cooperative navigation case study demonstrates that our OT-barycenter consensus outperforms an independent learners baseline in convergence speed and final coordination success.