mlsanityData to 7 Oct 2026

MAMBPO: Sample-efficient multi-robot reinforcement learning using learned world models

arXiv 2103.03662 ↗First posted 5 Mar 2021cs.RO cs.AI cs.MAcited by 16

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

Multi-robot systems can benefit from reinforcement learning (RL) algorithms that learn behaviours in a small number of trials, a property known as sample efficiency. This research thus investigates the use of learned world models to improve sample efficiency. We present a novel multi-agent model-based RL algorithm: Multi-Agent Model-Based Policy Optimization (MAMBPO), utilizing the Centralized Learning for Decentralized Execution (CLDE) framework. CLDE algorithms allow a group of agents to act in a fully decentralized manner after training. This is a desirable property for many systems comprising of multiple robots. MAMBPO uses a learned world model to improve sample efficiency compared to model-free Multi-Agent Soft Actor-Critic (MASAC). We demonstrate this on two simulated multi-robot tasks, where MAMBPO achieves a similar performance to MASAC, but requires far fewer samples to do so. Through this, we take an important step towards making real-life learning for multi-robot systems possible.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 11 of its 17 references that are in the map. First posted on arXiv 5 Mar 2021.

Builds on

11 of 17 references.

Closest ancestors · 2

Same direction, newest first

  1. 2019When to Trust Your Model: Model-Based Policy Optimization19 Jun
  2. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models30 May

Other references · 9

  1. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  2. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  3. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  4. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 259
  5. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 257
  6. 2019Benchmarking Model-Based Reinforcement Learningcited by 158
  7. 2017Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observabilitycited by 143
  8. 2017Regularization for Deep Learning: A Taxonomycited by 91
  9. 2019Multi-Agent Reinforcement Learning with Multi-Step Generative Modelscited by 18

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 3; 2023: 5; 2024: 3; 2025: 2.

16 papers.

Sort:

By area:

  1. 2021Model-Based Opponent Modelingcited by 21
  2. 2023A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problemcited by 18
  3. 2022Model-based Multi-agent Reinforcement Learning: Recent Progress and Prospectscited by 17
  4. 2021Centralized Model and Exploration Policy for Multi-Agent RLcited by 11
  5. 2024Decentralized Transformers with Centralized Aggregation are Sample-Efficient Multi-Agent World Modelscited by 10
  6. 2024Efficient Multi-agent Reinforcement Learning by Planningcited by 9
  7. 2022CLAS: Coordinating Multi-Robot Manipulation with Central Latent Action Spacescited by 6
  8. 2024CoDreamer: Communication-Based Decentralised World Modelscited by 5
  9. 2023Co-Learning Empirical Games and World Modelscited by 4
  10. 2021Nearest-Neighbor-based Collision Avoidance for Quadrotors via Reinforcement Learningcited by 3
  11. 2022Centralized Training with Hybrid Execution in Multi-Agent Reinforcement Learningcited by 2
  12. 2023Model-Based Decentralized Policy Optimizationcited by 1
4 more
  1. 2025Combining Bayesian Inference and Reinforcement Learning for Agent Decision Making: A Reviewcited by 1
  2. 2023An Algorithm For Adversary Aware Decentralized Networked MARLcited by 0
  3. 2023Formal Modelling for Multi-Robot Systems Under Uncertaintycited by 0
  4. 2025Toward Adaptable Multi-Agent Reinforcement Learning: An Assumption-Aware Reviewcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Scalable Multi-Agent Model-Based Reinforcement Learning
  2. 2025Empowering Multi-Robot Cooperation via Sequential World Models
  3. 2024Bi-CL: A Reinforcement Learning Framework for Robots Coordination Through Bi-level Optimization
  4. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Models
  5. 2026MA-WAM: Multi-Agent World-Action Model for Test-Time Planning
  6. 2024MAMBA: an Effective World Model Approach for Meta-Reinforcement Learning
  7. 2025Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics
  8. 2026MA-VLCM: A Vision Language Critic Model for Value Estimation of Policies in Multi-Agent Team Settings