mlsanityData to 7 Oct 2026

Mingling Foresight with Imagination: Model-Based Cooperative Multi-Agent Reinforcement Learning

arXiv 2204.09418 ↗First posted 20 Apr 2022cs.MA cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Recently, model-based agents have achieved better performance than model-free ones using the same computational budget and training time in single-agent environments. However, due to the complexity of multi-agent systems, it is tough to learn the model of the environment. The significant compounding error may hinder the learning process when model-based methods are applied to multi-agent tasks. This paper proposes an implicit model-based multi-agent reinforcement learning method based on value decomposition methods. Under this method, agents can interact with the learned virtual environment and evaluate the current state value according to imagined future states in the latent space, making agents have the foresight. Our approach can be applied to any multi-agent value decomposition method. The experimental results show that our method improves the sample efficiency in different partially observable Markov decision process domains.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 45 of its 69 references that are in the map. First posted on arXiv 20 Apr 2022.

Builds on

45 of 69 references.

References · 45

  1. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  4. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  6. 2020Mastering Atari with Discrete World Modelscited by 990
  7. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  8. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  9. 2019Solving Rubik's Cube with a Robot Handcited by 699
  10. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  11. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  12. 2019The StarCraft Multi-Agent Challengecited by 589
33 more
  1. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  2. 2019Model-Based Reinforcement Learning for Ataricited by 495
  3. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
  4. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
  5. 2016Learning Multiagent Communication with Backpropagationcited by 355
  6. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 259
  7. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 257
  8. 2018Model-Ensemble Trust-Region Policy Optimizationcited by 214
  9. 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
  10. 2019MAVEN: Multi-Agent Variational Explorationcited by 181
  11. 2020FACMAC: Factored Multi-Agent Centralised Policy Gradientscited by 178
  12. 2018Learning Attentional Communication for Multi-Agent Cooperationcited by 170
  13. 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 167
  14. 2015Learning Continuous Control Policies by Stochastic Value Gradientscited by 156
  15. 2018Model-Based Value Estimation for Efficient Model-Free Reinforcement Learningcited by 152
  16. 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 135
  17. 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
  18. 2017Value Prediction Networkcited by 117
  19. 2018Model-Based Reinforcement Learning via Meta-Policy Optimizationcited by 99
  20. 2017Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Gamescited by 93
  21. 2020Model-Based Multi-Agent RL in Zero-Sum Markov Games with Near-Optimal Sample Complexitycited by 91
  22. 2019Learning to Schedule Communication in Multi-agent Reinforcement Learningcited by 89
  23. 2018Learning and Querying Fast Generative Models for Reinforcement Learningcited by 48
  24. 2020Generative Temporal Difference Learning for Infinite-Horizon Predictioncited by 43
  25. 2019Learning to Combat Compounding-Error in Model-Based Reinforcement Learningcited by 39
  26. 2020Bidirectional Model-based Policy Optimizationcited by 38
  27. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  28. 2020Selective Dyna-style Planning Under Limited Model Capacitycited by 30
  29. 2021Model-based Multi-agent Policy Optimization with Adaptive Opponent-wise Rolloutscited by 24
  30. 2021HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanismcited by 19
  31. 2019Efficient and Robust Reinforcement Learning with Uncertainty-based Value Expansioncited by 13
  32. 2020Dynamic Horizon Value Estimation for Model-based Reinforcement Learningcited by 8
  33. 2020Model-based Multi-Agent Reinforcement Learning with Cooperative Prioritized Sweepingcited by 2

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1; 2025: 3.

6 papers.

  1. 2024Decentralized Transformers with Centralized Aggregation are Sample-Efficient Multi-Agent World Modelscited by 10
  2. 2025Learning Generalizable Skills from Offline Multi-Task Data for Multi-Agent Cooperationcited by 8
  3. 2023Adaptive parameter sharing for multi-agent reinforcement learningcited by 6
  4. 2023SEA: A Spatially Explicit Architecture for Multi-Agent Reinforcement Learningcited by 2
  5. 2025Ego-centric Learning of Communicative World Models for Autonomous Drivingcited by 2
  6. 2025Transformer World Model for Sample Efficient Multi-Agent Reinforcement Learningcited by 0