mlsanityData to 14 Oct 2026

VDFD: Multi-Agent Value Decomposition Framework with Disentangled World Model

arXiv 2309.04615 ↗First posted 8 Sep 2023cs.LG cs.AI cs.MAcited by 3

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

In this paper, we propose a novel model-based multi-agent reinforcement learning approach named Value Decomposition Framework with Disentangled World Model to address the challenge of achieving a common goal of multiple agents interacting in the same environment with reduced sample complexity. Due to scalability and non-stationarity problems posed by multi-agent systems, model-free methods rely on a considerable number of samples for training. In contrast, we use a modularized world model, composed of action-conditioned, action-free, and static branches, to unravel the complicated environment dynamics. Our model produces imagined outcomes based on past experience, without sampling directly from the real environment. We employ variational auto-encoders and variational graph auto-encoders to learn the latent representations for the world model, which is merged with a value-based framework to predict the joint action-value function and optimize the overall training objective. Experimental results on StarCraft II micro-management, Multi-Agent MuJoCo, and Level-Based Foraging challenges demonstrate that our method achieves high sample efficiency and exhibits superior performance compared to other baselines across a wide range of multi-agent learning tasks.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 52 of its 77 references that are in the map. First posted on arXiv 8 Sep 2023.

Builds on

52 of 77 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Mastering Diverse Domains through World Models10 Jan
  2. 2022A Survey on Model-based Reinforcement Learning19 Jun
  3. 2022Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Models27 May
  4. 2020Mastering Atari with Discrete World Models5 Oct
  5. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  6. 2019When to Trust Your Model: Model-Based Policy Optimization19 Jun
  7. 2018World Models27 Mar

Other references · 45

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2015Trust Region Policy Optimizationcited by 2,580
  5. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  6. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  8. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  9. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  10. 2019The StarCraft Multi-Agent Challengecited by 597
  11. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
  12. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
33 more
  1. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
  2. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
  3. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 270
  4. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 261
  5. 2019Recurrent Independent Mechanismscited by 219
  6. 2020Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscited by 212
  7. 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
  8. 2019MAVEN: Multi-Agent Variational Explorationcited by 185
  9. 2015Multiagent Cooperation and Competition with Deep Reinforcement Learningcited by 184
  10. 2020FACMAC: Factored Multi-Agent Centralised Policy Gradientscited by 178
  11. 2019GP-VAE: Deep Probabilistic Time Series Imputationcited by 172
  12. 2016Continuous Deep Q-Learning with Model-based Accelerationcited by 157
  13. 2015Incentivizing Exploration In Reinforcement Learning With Deep Predictive Modelscited by 154
  14. 2017Imagination-Augmented Agents for Deep Reinforcement Learningcited by 140
  15. 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 137
  16. 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
  17. 2020Shared Experience Actor-Critic for Multi-Agent Reinforcement Learningcited by 116
  18. 2020Provable Self-Play Algorithms for Competitive Reinforcement Learningcited by 113
  19. 2019Symmetric Graph Convolutional Autoencoder for Unsupervised Graph Representation Learningcited by 94
  20. 2020Model-Based Multi-Agent RL in Zero-Sum Markov Games with Near-Optimal Sample Complexitycited by 92
  21. 2015Massively Parallel Methods for Deep Reinforcement Learningcited by 82
  22. 2019Hierarchical Cooperative Multi-Agent Reinforcement Learning with Skill Discoverycited by 43
  23. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  24. 2022MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffercited by 34
  25. 2022Scalable Multi-Agent Model-Based Reinforcement Learningcited by 25
  26. 2021Model-based Multi-agent Policy Optimization with Adaptive Opponent-wise Rolloutscited by 24
  27. 2022Contrastive Identity-Aware Learning for Multi-Agent Value Decompositioncited by 19
  28. 2022Model-based Multi-agent Reinforcement Learning: Recent Progress and Prospectscited by 17
  29. 2020Model-based Policy Optimization with Unsupervised Model Adaptationcited by 16
  30. 2022Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computationcited by 14
  31. 2020Learning Implicit Credit Assignment for Cooperative Multi-Agent Reinforcement Learningcited by 10
  32. 2023Models as Agents: Optimizing Multi-Step Predictions of Interactive Local Models in Model-Based Multi-Agent Reinforcement Learningcited by 10
  33. 2023Self-Motivated Multi-Agent Explorationcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2026ActionParty: Multi-Subject Action Binding in Generative Video Gamescited by 5
  2. 2025Ego-centric Learning of Communicative World Models for Autonomous Drivingcited by 2
  3. 2025GenAI-based Multi-Agent Reinforcement Learning towards Distributed Agent Intelligence: A Generative-RL Agent Perspectivecited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Fixing Incomplete Value Function Decomposition for Multi-Agent Reinforcement Learning
  2. 2024Dynamic Deep Factor Graph for Multi-Agent Reinforcement Learning
  3. 2026Multi-Agent Model-Based Reinforcement Learning with Joint State-Action Learned Embeddings
  4. 2024Mixed Q-Functionals: Advancing Value-Based Methods in Cooperative MARL with Continuous Action Domains
  5. 2025Revisiting Multi-Agent World Modeling from a Diffusion-Inspired Perspective
  6. 2025Transformer World Model for Sample Efficient Multi-Agent Reinforcement Learning
  7. 2023Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularization
  8. 2023Boosting Value Decomposition via Unit-Wise Attentive State Representation for Cooperative Multi-Agent Reinforcement Learning