# Nucleolus Credit Assignment for Effective Coalitions in Multi-agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2503.00372 · HTML: https://mlsanity.com/paper/2503.00372/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 32 references that are in the map.
- First posted on arXiv 1 Mar 2025.

## Builds on (most cited first, 12 of 12)

- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2003.08839/) (2020)
- [Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network](https://mlsanity.com/paper/1808.03314/) (2018)
- [Reward Constrained Policy Optimization](https://mlsanity.com/paper/1805.11074/) (2018)
- [Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2006.10800/) (2020)
- [Emergent Complexity via Multi-Agent Competition](https://mlsanity.com/paper/1710.03748/) (2017)
- [Deep Coordination Graphs](https://mlsanity.com/paper/1910.00091/) (2019)
- [Shapley Q-value: A Local Reward Approach to Solve Global Reward Games](https://mlsanity.com/paper/1907.05707/) (2019)
- [Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition](https://mlsanity.com/paper/2105.08692/) (2021)
- [Variance-Constrained Actor-Critic Algorithms for Discounted and Average Reward MDPs](https://mlsanity.com/paper/1403.6530/) (2014)
- [SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learning](https://mlsanity.com/paper/2105.15013/) (2021)
- [ALMA: Hierarchical Learning for Composite Multi-Agent Tasks](https://mlsanity.com/paper/2205.14205/) (2022)

## Built on by (most cited first, 3 of 3)

- [Cooperative Game-Theoretic Credit Assignment for Multi-Agent Policy Gradients via the Core](https://mlsanity.com/paper/2506.04265/) (2025)
- [Toward Adaptable Multi-Agent Reinforcement Learning: An Assumption-Aware Review](https://mlsanity.com/paper/2507.10142/) (2025)
- [Who Gets the Reward & Who Gets the Blame? Evaluation-Aligned Training Signals for Multi-LLM Agents](https://mlsanity.com/paper/2511.10687/) (2025)

## Abstract (arXiv)

In cooperative multi-agent reinforcement learning (MARL), agents typically form a single grand coalition based on credit assignment to tackle a composite task, often resulting in suboptimal performance. This paper proposed a nucleolus-based credit assignment grounded in cooperative game theory, enabling the autonomous partitioning of agents into multiple small coalitions that can effectively identify and complete subtasks within a larger composite task. Specifically, our designed nucleolus Q-learning could assign fair credits to each agent, and the nucleolus Q-operator provides theoretical guarantees with interpretability for both learning convergence and the stability of the formed small coalitions. Through experiments on Predator-Prey and StarCraft scenarios across varying difficulty levels, our approach demonstrated the emergence of multiple effective coalitions during MARL training, leading to faster learning and superior performance in terms of win rate and cumulative rewards especially in hard and super-hard environments, compared to four baseline methods. Our nucleolus-based credit assignment showed the promise for complex composite tasks requiring effective subteams of agents.