# Stable and Efficient Shapley Value-Based Reward Reallocation for Multi-Agent Reinforcement Learning of Autonomous Vehicles

arXiv: https://arxiv.org/abs/2203.06333 · HTML: https://mlsanity.com/paper/2203.06333/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2022.

## Built on by (most cited first, 11 of 11)

- [Motion Planning for Autonomous Driving: The State of the Art and Future Perspectives](https://mlsanity.com/paper/2303.09824/) (2023)
- [Uncertainty Quantification of Collaborative Detection for Self-Driving](https://mlsanity.com/paper/2209.08162/) (2022)
- [Collaborative Multi-Object Tracking with Conformal Uncertainty Propagation](https://mlsanity.com/paper/2303.14346/) (2023)
- [What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?](https://mlsanity.com/paper/2212.02705/) (2022)
- [Spatial-Temporal-Aware Safe Multi-Agent Reinforcement Learning of Connected Autonomous Vehicles in Challenging Scenarios](https://mlsanity.com/paper/2210.02300/) (2022)
- [Efficiently Quantifying Individual Agent Importance in Cooperative MARL](https://mlsanity.com/paper/2312.08466/) (2023)
- [Multi Agent Path Finding using Evolutionary Game Theory](https://mlsanity.com/paper/2212.02010/) (2022)
- [High-order Interactions Modeling for Interpretable Multi-Agent Q-Learning](https://mlsanity.com/paper/2510.20218/) (2025)
- [Using Cooperative Game Theory to Prune Neural Networks](https://mlsanity.com/paper/2311.10468/) (2023)
- [Cooperative Reward Shaping for Multi-Agent Pathfinding](https://mlsanity.com/paper/2407.10403/) (2024)
- [Robust and Safe Multi-Agent Reinforcement Learning with Communication for Autonomous Vehicles: From Simulation to Hardware](https://mlsanity.com/paper/2506.00982/) (2025)

## Abstract (arXiv)

With the development of sensing and communication technologies in networked cyber-physical systems (CPSs), multi-agent reinforcement learning (MARL)-based methodologies are integrated into the control process of physical systems and demonstrate prominent performance in a wide array of CPS domains, such as connected autonomous vehicles (CAVs). However, it remains challenging to mathematically characterize the improvement of the performance of CAVs with communication and cooperation capability. When each individual autonomous vehicle is originally self-interest, we can not assume that all agents would cooperate naturally during the training process. In this work, we propose to reallocate the system's total reward efficiently to motivate stable cooperation among autonomous vehicles. We formally define and quantify how to reallocate the system's total reward to each agent under the proposed transferable utility game, such that communication-based cooperation among multi-agents increases the system's total reward. We prove that Shapley value-based reward reallocation of MARL locates in the core if the transferable utility game is a convex game. Hence, the cooperation is stable and efficient and the agents should stay in the coalition or the cooperating group. We then propose a cooperative policy learning algorithm with Shapley value reward reallocation. In experiments, compared with several literature algorithms, we show the improvement of the mean episode system reward of CAV systems using our proposed algorithm.