mlsanityData to 14 Oct 2026

MA-WAM: Multi-Agent World-Action Model for Test-Time Planning

arXiv 2609.31281 ↗First posted 25 Sep 2026cs.AIcited by 0

On the map: World models → Latent dynamics models and model-based RL also fits: World models for language agents

Abstract (arXiv)

Multi-agent cooperative tasks require different agents to execute a joint action simultaneously, and each agent's action affects both the observations and responses of the other agents. Hence, a world model is needed to predict the team return resulting from the joint actions of all agents. A naive extension directly applies a single-agent world model to each agent's action when predicting the team return step by step. However, such an extension fails to capture the dependencies among the simultaneous actions of multiple agents. We propose Multi-Agent World-Action Model (MA-WAM), a test-time planning framework that enables a frozen multi-agent flow policy to evaluate futures of candidate joint actions. To our knowledge, MA-WAM is the first test-time world-model planner for multi-agent flow policies. MA-WAM predicts the consequences of each joint action according to cross-agent dependencies and enables efficient candidate scoring. Across 30 offline multi-agent reinforcement learning (MARL) settings on MAMuJoCo, SMAC, and MPE, MA-WAM achieves mean relative gains of 22.0% over direct execution and 25.6% over uniform action selection. Under the standard evaluation protocol on an A100 GPU, MA-WAM adds 12.1 ms, accounting for 2.5% of the measured generation-and-scoring time.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 43 of its 49 references that are in the map. First posted on arXiv 25 Sep 2026.

Builds on

43 of 49 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026Mixture-of-World Models: Scaling Multi-Task Reinforcement Learning with Modular Latent Dynamics1 Feb
  2. 2026Puzzle it Out: Local-to-Global World Model for Offline Multi-Agent Reinforcement Learning12 Jan
  3. 2025Revisiting Multi-Agent World Modeling from a Diffusion-Inspired Perspective27 May
  4. 2025Improving Transformer World Models for Data-Efficient RL3 Feb
  5. 2024Decentralized Transformers with Centralized Aggregation are Sample-Efficient Multi-Agent World Models22 Jun
  6. 2024Diffusion for World Modeling: Visual Details Matter in Atari20 May
  7. 2023TD-MPC2: Scalable, Robust World Models for Continuous Control25 Oct
  8. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec

Other references · 35

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  2. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  5. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  6. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
  7. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
  8. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  9. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  10. 2024Genie: Generative Interactive Environmentscited by 747
  11. 2019The StarCraft Multi-Agent Challengecited by 597
  12. 2025Mean Flows for One-step Generative Modelingcited by 561
23 more
  1. 2024One Step Diffusion via Shortcut Modelscited by 391
  2. 2023From Sparse to Soft Mixtures of Expertscited by 248
  3. 2025Flow Q-Learningcited by 184
  4. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 182
  5. 2020FACMAC: Factored Multi-Agent Centralised Policy Gradientscited by 178
  6. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 155
  7. 2021Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learningcited by 79
  8. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RLcited by 78
  9. 2023MADiff: Offline Multi-agent Learning with Diffusion Modelscited by 75
  10. 2021Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectificationcited by 61
  11. 2025RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Modelscited by 61
  12. 2025MindJourney: Test-Time Scaling with World Models for Spatial Reasoningcited by 50
  13. 2025Monte Carlo Tree Diffusion for System 2 Planningcited by 33
  14. 2021Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Taskscited by 32
  15. 2025Scaling Offline RL via Efficient and Expressive Shortcut Modelscited by 27
  16. 2023Improving Generalization and Data Efficiency with Diffusion in Offline Multi-agent RLcited by 15
  17. 2024Offline Multi-Agent Reinforcement Learning via In-Sample Sequential Policy Optimizationcited by 15
  18. 2025Multi-agent Coordination via Flow Matchingcited by 10
  19. 2023Off-the-Grid MARL: Datasets with Baselines for Offline Multi-Agent Reinforcement Learningcited by 8
  20. 2024ComaDICE: Offline Cooperative Multi-Agent Reinforcement Learning with Stationary Distribution Shift Regularizationcited by 8
  21. 2025Mixture-of-Experts Meets In-Context Reinforcement Learningcited by 8
  22. 2026Value-Guidance MeanFlow for Offline Multi-Agent Reinforcement Learningcited by 3
  23. 2025Unifying Agent Interaction and World Information for Multi-agent Coordinationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026G2MAF: Test-Time Gradient Guidance for Multi-Agent Flow Policies
  2. 2026MA-FPPO: Multi-Agent Flow-Pretrained Policy Optimization
  3. 2025An Extended Benchmarking of Multi-Agent Reinforcement Learning Algorithms in Complex Fully Cooperative Tasks
  4. 2025FutureWeaver: Planning Test-Time Compute for Multi-Agent Systems with Modularized Collaboration
  5. 2026Merging model-based control with multi-agent reinforcement learning for multi-agent cooperative teaming strategies
  6. 2025Transformer World Model for Sample Efficient Multi-Agent Reinforcement Learning
  7. 2025Empowering Multi-Robot Cooperation via Sequential World Models
  8. 2021MAMBPO: Sample-efficient multi-robot reinforcement learning using learned world models