mlsanityData to 7 Oct 2026

Remember and Forget Experience Replay for Multi-Agent Reinforcement Learning

arXiv 2203.13319 ↗First posted 24 Mar 2022cs.LG cs.AI math.OCcited by 2

Not on the map. Why?

Abstract (arXiv)

We present the extension of the Remember and Forget for Experience Replay (ReF-ER) algorithm to Multi-Agent Reinforcement Learning (MARL). ReF-ER was shown to outperform state of the art algorithms for continuous control in problems ranging from the OpenAI Gym to complex fluid flows. In MARL, the dependencies between the agents are included in the state-value estimator and the environment dynamics are modeled via the importance weights used by ReF-ER. In collaborative environments, we find the best performance when the value is estimated using individual rewards and we ignore the effects of other actions on the transition map. We benchmark the performance of ReF-ER MARL on the Stanford Intelligent Systems Laboratory (SISL) environments. We find that employing a single feed-forward neural network for the policy and the value function in ReF-ER MARL, outperforms state of the art algorithms that rely on complex neural network architectures.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 26 of its 42 references that are in the map. First posted on arXiv 24 Mar 2022.

Builds on

26 of 42 references.

References · 26

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2015Trust Region Policy Optimizationcited by 2,503
  6. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  7. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  8. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  9. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  10. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  11. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
  12. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
14 more
  1. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  2. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
  3. 2016Safe and Efficient Off-Policy Reinforcement Learningcited by 260
  4. 2018Distributed Prioritized Experience Replaycited by 260
  5. 2017Learning with Opponent-Learning Awarenesscited by 218
  6. 2020PettingZoo: Gym for Multi-Agent Reinforcement Learningcited by 209
  7. 2016Sample Efficient Actor-Critic with Experience Replaycited by 185
  8. 2020FACMAC: Factored Multi-Agent Centralised Policy Gradientscited by 178
  9. 2020Shared Experience Actor-Critic for Multi-Agent Reinforcement Learningcited by 114
  10. 2018Efficient collective swimming by harnessing vortices through deep reinforcement learningcited by 73
  11. 2021Scientific multi-agent reinforcement learning for wall-models of turbulent flowscited by 52
  12. 2018Remember and Forget for Experience Replaycited by 42
  13. 2018Clipped Action Policy Gradientcited by 17
  14. 2019SMIX(λ): Enhancing Centralized Value Functions for Cooperative Multi-Agent Reinforcement Learningcited by 15

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1.

2 papers.

  1. 2022Universally Expressive Communication in Multi-Agent Reinforcement Learningcited by 2
  2. 2023Discovering Individual Rewards in Collective Behavior through Inverse Multi-Agent Reinforcement Learningcited by 2