# Exploration with Unreliable Intrinsic Reward in Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/1906.02138 · HTML: https://mlsanity.com/paper/1906.02138/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2019.

## Built on by (most cited first, 13 of 13)

- [Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain](https://mlsanity.com/paper/2109.06668/) (2021)
- [Solving Continuous Control via Q-learning](https://mlsanity.com/paper/2210.12566/) (2022)
- [ELIGN: Expectation Alignment as a Multi-Agent Intrinsic Reward](https://mlsanity.com/paper/2210.04365/) (2022)
- [Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolution](https://mlsanity.com/paper/2404.04253/) (2024)
- [Cooperative and Competitive Biases for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2101.06890/) (2021)
- [Collaborative Training of Heterogeneous Reinforcement Learning Agents in Environments with Sparse Rewards: What and When to Share?](https://mlsanity.com/paper/2202.12174/) (2022)
- [Strategically Efficient Exploration in Competitive Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2107.14698/) (2021)
- [A Survey of Historical Learning: Learning Models with Learning History](https://mlsanity.com/paper/2303.12992/) (2023)
- [Credit Assignment with Meta-Policy Gradient for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2102.12957/) (2021)
- [Curiosity-Driven Multi-Agent Exploration with Mixed Objectives](https://mlsanity.com/paper/2210.16468/) (2022)
- [Learning Individual Intrinsic Reward in Multi-Agent Reinforcement Learning via Incorporating Generalized Human Expertise](https://mlsanity.com/paper/2507.18867/) (2025)
- [SVDE: Scalable Value-Decomposition Exploration for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2303.09058/) (2023)
- [Optimistic ε-Greedy Exploration for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2502.03506/) (2025)

## Abstract (arXiv)

This paper investigates the use of intrinsic reward to guide exploration in multi-agent reinforcement learning. We discuss the challenges in applying intrinsic reward to multiple collaborative agents and demonstrate how unreliable reward can prevent decentralized agents from learning the optimal policy. We address this problem with a novel framework, Independent Centrally-assisted Q-learning (ICQL), in which decentralized agents share control and an experience replay buffer with a centralized agent. Only the centralized agent is intrinsically rewarded, but the decentralized agents still benefit from improved exploration, without the distraction of unreliable incentives.