mlsanityData to 14 Oct 2026

Extending NGU to Multi-Agent RL: A Preliminary Study

arXiv 2512.01321 ↗First posted 1 Dec 2025cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The Never Give Up (NGU) algorithm has proven effective in reinforcement learning tasks with sparse rewards by combining episodic novelty and intrinsic motivation. In this work, we extend NGU to multi-agent environments and evaluate its performance in the simple_tag environment from the PettingZoo suite. Compared to a multi-agent DQN baseline, NGU achieves moderately higher returns and more stable learning dynamics. We investigate three design choices: (1) shared replay buffer versus individual replay buffers, (2) sharing episodic novelty among agents using different k thresholds, and (3) using heterogeneous values of the beta parameter. Our results show that NGU with a shared replay buffer yields the best performance and stability, highlighting that the gains come from combining NGU intrinsic exploration with experience sharing. Novelty sharing performs comparably when k = 1 but degrades learning for larger values. Finally, heterogeneous beta values do not improve over a small common value. These findings suggest that NGU can be effectively applied in multi-agent settings when experiences are shared and intrinsic exploration signals are carefully tuned.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 25 references that are in the map. First posted on arXiv 1 Dec 2025.

Builds on

16 of 25 references.

References · 16

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  3. 2018Exploration by Random Network Distillationcited by 993
  4. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  5. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  6. 2019The StarCraft Multi-Agent Challengecited by 597
  7. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
  8. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 309
  9. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 212
  10. 2020Never Give Up: Learning Directed Exploration Strategiescited by 212
  11. 2018Learning Montezuma's Revenge from a Single Demonstrationcited by 80
  12. 2021Episodic Multi-agent Reinforcement Learning with Curiosity-Driven Explorationcited by 49
4 more
  1. 2021Deep reinforcement learning in medical imaging: A literature reviewcited by 34
  2. 2019Coordinated Exploration via Intrinsic Rewards for Multi-Agent Reinforcement Learningcited by 30
  3. 2024Settling Decentralized Multi-Agent Coordinated Exploration by Novelty Sharingcited by 6
  4. 2024Efficient Episodic Memory Utilization of Cooperative Multi-Agent Reinforcement Learningcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025MIR: Efficient Exploration in Episodic Multi-Agent Reinforcement Learning via Mutual Intrinsic Reward
  2. 2025From Solo to Symphony: Orchestrating Multi-Agent Collaboration with Single-Agent Demos
  3. 20252048: Reinforcement Learning in a Delayed Reward Environment
  4. 2024Novelty-Guided Data Reuse for Efficient and Diversified Multi-Agent Reinforcement Learning
  5. 2024Joint Intrinsic Motivation for Coordinated Exploration in Multi-Agent Deep Reinforcement Learning
  6. 2024Laser Learning Environment: A new environment for coordination-critical multi-agent tasks
  7. 2026Reward-free Pretraining for Reinforcement Learning via Occupancy Coverage Maximization
  8. 2025Simulus: Combining Improvements in Sample-Efficient World Model Agents