# Effective and Stable Role-Based Multi-Agent Collaboration by Structural Information Principles

arXiv: https://arxiv.org/abs/2304.00755 · HTML: https://mlsanity.com/paper/2304.00755/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 40 references that are in the map.
- First posted on arXiv 3 Apr 2023.

## Builds on (most cited first, 14 of 14)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1905.05408/) (2019)
- [Emergent Tool Use From Multi-Agent Autocurricula](https://mlsanity.com/paper/1909.07528/) (2019)
- [QPLEX: Duplex Dueling Multi-Agent Q-Learning](https://mlsanity.com/paper/2008.01062/) (2020)
- [MAVEN: Multi-Agent Variational Exploration](https://mlsanity.com/paper/1910.07483/) (2019)
- [Multiagent Cooperation and Competition with Deep Reinforcement Learning](https://mlsanity.com/paper/1511.08779/) (2015)
- [ROMA: Multi-Agent Reinforcement Learning with Emergent Roles](https://mlsanity.com/paper/2003.08039/) (2020)
- [Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications](https://mlsanity.com/paper/1812.11794/) (2018)
- [RODE: Learning Roles to Decompose Multi-Agent Tasks](https://mlsanity.com/paper/2010.01523/) (2020)
- [Game-Theoretic Multiagent Reinforcement Learning](https://mlsanity.com/paper/2011.00583/) (2020)
- [Automating DBSCAN via Deep Reinforcement Learning](https://mlsanity.com/paper/2208.04537/) (2022)

## Built on by (most cited first, 12 of 12)

- [SeBot: Structural Entropy Guided Multi-View Contrastive Learning for Social Bot Detection](https://mlsanity.com/paper/2405.11225/) (2024)
- [Hierarchical State Abstraction Based on Structural Information Principles](https://mlsanity.com/paper/2304.12000/) (2023)
- [MultiSPANS: A Multi-range Spatial-Temporal Transformer Network for Traffic Forecast via Structural Entropy Optimization](https://mlsanity.com/paper/2311.02880/) (2023)
- [Adversarial Socialbots Modeling Based on Structural Information Principles](https://mlsanity.com/paper/2312.08098/) (2023)
- [Unsupervised Skin Lesion Segmentation via Structural Entropy Minimization on Multi-Scale Superpixel Graphs](https://mlsanity.com/paper/2309.01899/) (2023)
- [Kaleidoscope: Learnable Masks for Heterogeneous Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2410.08540/) (2024)
- [Effective Exploration Based on the Structural Information Principles](https://mlsanity.com/paper/2410.06621/) (2024)
- [Structural Information-based Hierarchical Diffusion for Offline Reinforcement Learning](https://mlsanity.com/paper/2509.21942/) (2025)
- [Retaining Suboptimal Actions to Follow Shifting Optima in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2602.17062/) (2026)
- [Structural Entropy Guided Probabilistic Coding](https://mlsanity.com/paper/2412.08841/) (2024)
- [Adaptive and Robust DBSCAN with Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2505.04339/) (2025)
- [Semi-Supervised Clustering via Structural Entropy with Different Constraints](https://mlsanity.com/paper/2312.10917/) (2023)

## Abstract (arXiv)

Role-based learning is a promising approach to improving the performance of Multi-Agent Reinforcement Learning (MARL). Nevertheless, without manual assistance, current role-based methods cannot guarantee stably discovering a set of roles to effectively decompose a complex task, as they assume either a predefined role structure or practical experience for selecting hyperparameters. In this article, we propose a mathematical Structural Information principles-based Role Discovery method, namely SIRD, and then present a SIRD optimizing MARL framework, namely SR-MARL, for multi-agent collaboration. The SIRD transforms role discovery into a hierarchical action space clustering. Specifically, the SIRD consists of structuralization, sparsification, and optimization modules, where an optimal encoding tree is generated to perform abstracting to discover roles. The SIRD is agnostic to specific MARL algorithms and flexibly integrated with various value function factorization approaches. Empirical evaluations on the StarCraft II micromanagement benchmark demonstrate that, compared with state-of-the-art MARL algorithms, the SR-MARL framework improves the average test win rate by 0.17%, 6.08%, and 3.24%, and reduces the deviation by 16.67%, 30.80%, and 66.30%, under easy, hard, and super hard scenarios.