mlsanityData to 13 Oct 2026

Never Explore Repeatedly in Multi-Agent Reinforcement Learning

arXiv 2308.09909 ↗First posted 19 Aug 2023cs.LG cs.AI cs.MAcited by 1

Not on the map. Why?

Abstract (arXiv)

In the realm of multi-agent reinforcement learning, intrinsic motivations have emerged as a pivotal tool for exploration. While the computation of many intrinsic rewards relies on estimating variational posteriors using neural network approximators, a notable challenge has surfaced due to the limited expressive capability of these neural statistics approximators. We pinpoint this challenge as the "revisitation" issue, where agents recurrently explore confined areas of the task space. To combat this, we propose a dynamic reward scaling approach. This method is crafted to stabilize the significant fluctuations in intrinsic rewards in previously explored areas and promote broader exploration, effectively curbing the revisitation phenomenon. Our experimental findings underscore the efficacy of our approach, showcasing enhanced performance in demanding environments like Google Research Football and StarCraft II micromanagement tasks, especially in sparse reward settings.

Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Builds on 52 of its 60 references that are in the map. First posted on arXiv 19 Aug 2023.

Builds on

52 of 60 references.

References · 52

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,753
  2. 2016Learning without Forgettingcited by 2,479
  3. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,323
  4. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,525
  5. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,180
  6. 2018Experience Replay for Continual Learningcited by 1,082
  7. 2018Exploration by Random Network Distillationcited by 971
  8. 2016Deep Variational Information Bottleneckcited by 924
  9. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 910
  10. 2017Counterfactual Multi-Agent Policy Gradientscited by 739
  11. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 700
  12. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
38 more
  1. 2019The StarCraft Multi-Agent Challengecited by 592
  2. 2018Progress & Compress: A scalable framework for continual learningcited by 516
  3. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 307
  4. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 286
  5. 2018Large-Scale Study of Curiosity-Driven Learningcited by 279
  6. 2019Continual learning with hypernetworkscited by 268
  7. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 268
  8. 2019Self-Supervised Exploration via Disagreementcited by 264
  9. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 258
  10. 2017Count-Based Exploration with Neural Density Modelscited by 255
  11. 2018Selective Experience Replay for Lifelong Learningcited by 240
  12. 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 219
  13. 2020Never Give Up: Learning Directed Exploration Strategiescited by 211
  14. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 208
  15. 2019The Hanabi Challenge: A New Frontier for AI Researchcited by 194
  16. 2019MAVEN: Multi-Agent Variational Explorationcited by 183
  17. 2017Deep Exploration via Randomized Value Functionscited by 155
  18. 2020RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environmentscited by 135
  19. 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 135
  20. 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
  21. 2018Count-Based Exploration with the Successor Representationcited by 112
  22. 2021Celebrating Diversity in Shared Multi-Agent Reinforcement Learningcited by 96
  23. 2019Keeping Your Distance: Solving Sparse Reward Tasks Using Self-Balancing Shaped Rewardscited by 76
  24. 2019Influence-Based Multi-Agent Explorationcited by 75
  25. 2018Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learningcited by 68
  26. 2020My Body is a Cage: the Role of Morphology in Graph-Based Incompatible Controlcited by 68
  27. 2021Cooperative Exploration for Multi-Agent Deep Reinforcement Learningcited by 57
  28. 2021Episodic Multi-agent Reinforcement Learning with Curiosity-Driven Explorationcited by 49
  29. 2016Q(λ) with Off-Policy Correctionscited by 48
  30. 2017Count-Based Exploration in Feature Space for Reinforcement Learningcited by 48
  31. 2020Emergent Social Learning via Multi-agent Reinforcement Learningcited by 44
  32. 2020See, Hear, Explore: Curiosity via Audio-Visual Associationcited by 43
  33. 2020UneVEn: Universal Value Exploration for Multi-Agent Reinforcement Learningcited by 32
  34. 2020Lifelong Policy Gradient Learning of Factored Policies for Faster Training Without Forgettingcited by 29
  35. 2020Exploration in Approximate Hyper-State Space for Meta Reinforcement Learningcited by 28
  36. 2022Learning Bayesian Sparse Networks with Full Experience Replay for Continual Learningcited by 27
  37. 2020The Emergence of Individualitycited by 22
  38. 2019SMIX(λ): Enhancing Centralized Value Functions for Cooperative Multi-Agent Reinforcement Learningcited by 15

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Multi-Sender Persuasion: A Computational Perspectivecited by 9