mlsanityData to 7 Oct 2026

Settling Decentralized Multi-Agent Coordinated Exploration by Novelty Sharing

arXiv 2402.02097 ↗First posted 3 Feb 2024cs.MA cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Exploration in decentralized cooperative multi-agent reinforcement learning faces two challenges. One is that the novelty of global states is unavailable, while the novelty of local observations is biased. The other is how agents can explore in a coordinated way. To address these challenges, we propose MACE, a simple yet effective multi-agent coordinated exploration method. By communicating only local novelty, agents can take into account other agents' local novelty to approximate the global novelty. Further, we newly introduce weighted mutual information to measure the influence of one agent's action on other agents' accumulated novelty. We convert it as an intrinsic reward in hindsight to encourage agents to exert more influence on other agents' exploration and boost coordinated exploration. Empirically, we show that MACE achieves superior performance in three multi-agent environments with sparse rewards.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 41 of its 52 references that are in the map. First posted on arXiv 3 Feb 2024.

Builds on

41 of 52 references.

References · 41

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016OpenAI Gymcited by 1,684
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  4. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  5. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  6. 2018Exploration by Random Network Distillationcited by 966
  7. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  8. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  9. 2019The StarCraft Multi-Agent Challengecited by 589
  10. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  11. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  12. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 345
29 more
  1. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 307
  2. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 281
  3. 2018Large-Scale Study of Curiosity-Driven Learningcited by 277
  4. 2019Self-Supervised Exploration via Disagreementcited by 263
  5. 2020Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscited by 207
  6. 2018TarMAC: Targeted Multi-Agent Communicationcited by 201
  7. 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 189
  8. 2019Efficient Exploration via State Marginal Matchingcited by 186
  9. 2019MAVEN: Multi-Agent Variational Explorationcited by 181
  10. 2018Learning Attentional Communication for Multi-Agent Cooperationcited by 170
  11. 2020RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environmentscited by 135
  12. 2021V-Learning -- A Simple, Efficient, Decentralized Algorithm for Multiagent RLcited by 111
  13. 2021Celebrating Diversity in Shared Multi-Agent Reinforcement Learningcited by 96
  14. 2019Learning to Schedule Communication in Multi-agent Reinforcement Learningcited by 89
  15. 2019Learning Nearly Decomposable Value Functions Via Communication Minimizationcited by 85
  16. 2019Influence-Based Multi-Agent Explorationcited by 75
  17. 2019Learning Efficient Multi-agent Communication: An Information Bottleneck Approachcited by 60
  18. 2020Learning Individually Inferred Communication for Multi-Agent Cooperationcited by 55
  19. 2021Episodic Multi-agent Reinforcement Learning with Curiosity-Driven Explorationcited by 49
  20. 2021Adversarially Guided Actor-Criticcited by 47
  21. 2022The Complexity of Markov Equilibrium in Stochastic Gamescited by 45
  22. 2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
  23. 2022MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffercited by 34
  24. 2019Coordinated Exploration via Intrinsic Rewards for Multi-Agent Reinforcement Learningcited by 30
  25. 2022PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaborationcited by 23
  26. 2020The Emergence of Individualitycited by 22
  27. 2020A Maximum Mutual Information Framework for Multi-Agent Reinforcement Learningcited by 11
  28. 2022Exploring through Random Curiosity with General Value Functionscited by 10
  29. 2021LINDA: Multi-Agent Local Information Decomposition for Awareness of Teammatescited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025Wonder Wins Ways: Curiosity-Driven Exploration through Multi-Agent Contextual Calibrationcited by 4
  2. 2025Contextual Knowledge Sharing in Multi-Agent Reinforcement Learning with Decentralized Communication and Coordinationcited by 2
  3. 2025Focusing Influence Mechanism for Multi-Agent Reinforcement Learningcited by 2
  4. 2025Extending NGU to Multi-Agent RL: A Preliminary Studycited by 0
  5. 2026Large Language Models Explore by Latent Distillingcited by 0
  6. 2026MAGIC: Multi-Step Advantage-Gated Causal Influence for Multi-agent Reinforcement Learningcited by 0