Learning Individual Policies in Large Multi-agent Systems through Local Variance Minimization
Not on the map. Why?
Abstract (arXiv)
In multi-agent systems with large number of agents, typically the contribution of each agent to the value of other agents is minimal (e.g., aggregation systems such as Uber, Deliveroo). In this paper, we consider such multi-agent systems where each agent is self-interested and takes a sequence of decisions and represent them as a Stochastic Non-atomic Congestion Game (SNCG). We derive key properties for equilibrium solutions in SNCG model with non-atomic and also nearly non-atomic agents. With those key equilibrium properties, we provide a novel Multi-Agent Reinforcement Learning (MARL) mechanism that minimizes variance across values of agents in the same state. To demonstrate the utility of this new mechanism, we provide detailed results on a real-world taxi dataset and also a generic simulator for aggregation systems. We show that our approach reduces the variance in revenues earned by taxi drivers, while still providing higher joint revenues than leading approaches.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 53 references that are in the map. First posted on arXiv 27 Dec 2022.
Builds on
16 of 53 references.
References · 16
- 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
- 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
- 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
- 2018Mean Field Multi-Agent Reinforcement Learningcited by 207
- 2016Deep Reinforcement Learning from Self-Play in Imperfect-Information Gamescited by 158
- 2017Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learningcited by 117
- 2021Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharingcited by 87
- 2021Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learningcited by 73
- 2018Deep Reinforcement Learning for Swarm Systemscited by 45
- 2019Neural Approximate Dynamic Programming for On-Demand Ride-Poolingcited by 33
- 2020Multi Type Mean Field Reinforcement Learningcited by 26
- 2021Decentralized Mean Field Gamescited by 12
4 more
- 2021Variance Penalized On-Policy and Off-Policy Actor-Criticcited by 10
- 2020Stochastic Potential Gamescited by 4
- 2018Entropy based Independent Learning in Anonymous Multi-Agent Settingscited by 2
- 2014Learning Nash Equilibria in Congestion Gamescited by 1
Built on by
Not cited yet.