Graph-SND: Sparse Aggregation for Behavioral Diversity in Multi-Agent Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
System Neural Diversity (SND) measures behavioral heterogeneity in multi-agent reinforcement learning by averaging pairwise distances over all binomn2 agent pairs, making each call quadratic in team size. We introduce Graph-SND, which replaces this complete-graph average with a weighted average over the edges of an arbitrary graph G. Three regimes follow: G=Kn recovers SND exactly; a fixed sparse G defines a localized diversity measure at O(|E|) cost; and random edge samples yield an unbiased Horvitz-Thompson estimator and a normalized sample mean with O(1/√m) concentration in the sampled edge count m. For fixed sparse graphs we prove forwarding-index distortion bounds for expanders and a spectral refinement under low-rank distance structure; for random d-regular graphs we prove an unconditional probabilistic widetildeO(Dmax/√n) bound. On VMAS we verify recovery, unbiasedness, concentration, and wall-clock scaling, with a PettingZoo TVD panel checking non-Gaussian transfer. In a 500-iteration n=100 PPO run, Bernoulli-0.1 Graph-SND tracks full SND while reducing per-call metric time by about 10×, and frozen-policy GPU timing up to n=500 follows the predicted binomn2/|E| speedup. Random d-regular expanders empirically achieve SNDGu/SND ∈ [0.9987, 1.0013] at Theta(n log n) edges. In DiCo diversity control at n=50, Bernoulli-0.1 Graph-SND preserves set-point tracking with paired reward differences indistinguishable from zero across nine matched cells while cutting per-call metric cost by ~9.5×. Together, these results show that the SND aggregation bottleneck can be removed without changing the metric's semantics, yielding a drop-in sparse alternative that scales beyond complete-graph SND and supports both passive measurement and closed-loop diversity control.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 14 of its 32 references that are in the map. First posted on arXiv 6 May 2026.
Builds on
14 of 32 references.
References · 14
- 2016Learning Multiagent Communication with Backpropagationcited by 355
- 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 135
- 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
- 2020Effective Diversity in Population Based Reinforcement Learningcited by 116
- 2022VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learningcited by 63
- 2023Heterogeneous Multi-Robot Reinforcement Learningcited by 34
- 2020The Emergence of Adversarial Communication in Multi-Agent Reinforcement Learningcited by 28
- 2021Quantifying the effects of environment and population diversity in multi-agent reinforcement learningcited by 21
- 2023System Neural Diversity: Measuring Behavioral Heterogeneity in Multi-Agent Learningcited by 12
- 2022Policy Diagnosis via Measuring Role Diversity in Cooperative Multi-agent RLcited by 11
- 2024Controlling Behavioral Diversity in Multi-Agent Reinforcement Learningcited by 10
- 2024HyperMARL: Adaptive Hypernetworks for Multi-Agent RLcited by 10
2 more
Built on by
Not cited yet.