# A Unified Framework for Locality in Scalable MARL

arXiv: https://arxiv.org/abs/2602.16966 · HTML: https://mlsanity.com/paper/2602.16966/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 17 references that are in the map.
- First posted on arXiv 19 Feb 2026.

## Builds on (most cited first, 6 of 6)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [A Theory of Regularized Markov Decision Processes](https://mlsanity.com/paper/1901.11275/) (2019)
- [Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents](https://mlsanity.com/paper/1802.08757/) (2018)
- [Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Reward](https://mlsanity.com/paper/2006.06626/) (2020)
- [Multi-Agent Reinforcement Learning in Stochastic Networked Systems](https://mlsanity.com/paper/2006.06555/) (2020)

## Built on by (most cited first, 1 of 1)

- [Scalable Policy Optimization for Networked Multi-Agent Reinforcement Learning with Continuous State-Action Spaces](https://mlsanity.com/paper/2607.18554/) (2026)

## Abstract (arXiv)

Scalable methods for networked multi-agent reinforcement learning let each agent plan using only a small neighborhood of the agent graph. This works only when the system is value-local, meaning a perturbation at one agent affects the long-run value at another agent weakly when the two are far apart. In the average-reward setting, the standard way to certify locality is the Dobrushin row-sum bound on a single matrix Cπ that captures how each agent's next state depends on each other agent's current state. To make this matrix easy to work with, prior work bounds it by a supremum over joint actions. The resulting bound is independent of the policy, but it is loose whenever the policy never picks the worst-case action. We split Cπ into pieces that separately track environment sensitivity and policy sensitivity, Cπ preceq Emathrm s+Emathrm aPi(π), where Emathrm s measures how the next state moves with the current state, Emathrm a measures how it moves with the current action, and Pi(π) measures how reactive the policy is to changes in state. The spectral radius of Hπ := Emathrm s+Emathrm aPi(π) then controls the decay of the average-reward Poisson solution, and the spectral certificate ρ(Hπ)<1 is strictly weaker than the row-sum condition |Hπ|_∞<1 on the same matrix and applies in regimes where policy-independent action-supremum bounds used in prior Dobrushin-style work cannot. For temperature-τ softmax policies we get Pi(π)≤ L/(2τ), so the softmax temperature directly controls locality. We use this decay result to give a deterministic oracle guarantee for a block-coordinate KL-proximal policy-improvement template whose truncation bias decays exponentially in the message-passing radius kappa.