Graph Exploration for Effective Multi-agent Q-Learning
Not on the map. Why?
Abstract (arXiv)
This paper proposes an exploration technique for multi-agent reinforcement learning (MARL) with graph-based communication among agents. We assume the individual rewards received by the agents are independent of the actions by the other agents, while their policies are coupled. In the proposed framework, neighbouring agents collaborate to estimate the uncertainty about the state-action space in order to execute more efficient explorative behaviour. Different from existing works, the proposed algorithm does not require counting mechanisms and can be applied to continuous-state environments without requiring complex conversion techniques. Moreover, the proposed scheme allows agents to communicate in a fully decentralized manner with minimal information exchange. And for continuous-state scenarios, each agent needs to exchange only a single parameter vector. The performance of the algorithm is verified with theoretical results for discrete-state scenarios and with experiments for continuous ones.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 18 of its 42 references that are in the map. First posted on arXiv 19 Apr 2023.
Builds on
18 of 42 references.
References · 18
- 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
- 2016Deep Exploration via Bootstrapped DQNcited by 518
- 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
- 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
- 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
- 2018Mean Field Multi-Agent Reinforcement Learningcited by 202
- 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
- 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
- 2017Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learningcited by 114
- 2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
- 2021Cooperative Exploration for Multi-Agent Deep Reinforcement Learningcited by 57
- 2021Ensemble Bootstrapping for Q-Learningcited by 30
6 more
- 2022Learning Infinite-Horizon Average-Reward Markov Decision Processes with Constraintscited by 29
- 2021Provably Efficient Cooperative Multi-Agent Reinforcement Learning with Function Approximationcited by 23
- 2020Temporal Difference Uncertainties as a Signal for Explorationcited by 15
- 2021Provably Efficient Multi-Agent Reinforcement Learning with Fully Decentralized Communicationcited by 5
- 2023Reinforcement Learning with Function Approximation: From Linear to Nonlinearcited by 4
- 2019ISL: A novel approach for deep explorationcited by 1
Built on by
Citing papers per year: 2024: 1.
1 paper.