mlsanityData to 7 Oct 2026

Centralized Training with Hybrid Execution in Multi-Agent Reinforcement Learning

arXiv 2210.06274 ↗First posted 12 Oct 2022cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

We introduce hybrid execution in multi-agent reinforcement learning (MARL), a new paradigm in which agents aim to successfully complete cooperative tasks with arbitrary communication levels at execution time by taking advantage of information-sharing among the agents. Under hybrid execution, the communication level can range from a setting in which no communication is allowed between agents (fully decentralized), to a setting featuring full communication (fully centralized), but the agents do not know beforehand which communication level they will encounter at execution time. To formalize our setting, we define a new class of multi-agent partially observable Markov decision processes (POMDPs) that we name hybrid-POMDPs, which explicitly model a communication process between the agents. We contribute MARO, an approach that makes use of an auto-regressive predictive model, trained in a centralized manner, to estimate missing agents' observations at execution time. We evaluate MARO on standard scenarios and extensions of previous benchmarks tailored to emphasize the negative impact of partial observability in MARL. Experimental results show that our method consistently outperforms relevant baselines, allowing agents to act with faulty communication while successfully exploiting shared information.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 29 of its 47 references that are in the map. First posted on arXiv 12 Oct 2022.

Builds on

29 of 47 references.

References · 29

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  4. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  5. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  6. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  7. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 373
  8. 2016Learning Multiagent Communication with Backpropagationcited by 355
  9. 2020Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscited by 207
  10. 2018Learning Attentional Communication for Multi-Agent Cooperationcited by 170
  11. 2017Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observabilitycited by 143
  12. 2018Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Taskscited by 114
17 more
  1. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
  2. 2016Opponent Modeling in Deep Reinforcement Learningcited by 94
  3. 2020Learning Latent Representations to Influence Multi-Agent Interactioncited by 92
  4. 2019Learning to Schedule Communication in Multi-agent Reinforcement Learningcited by 89
  5. 2019A Survey on Traffic Signal Control Methodscited by 78
  6. 2019Learning Efficient Multi-agent Communication: An Information Bottleneck Approachcited by 60
  7. 2020Agent Modelling under Partial Observability for Deep Reinforcement Learningcited by 56
  8. 2019Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Controlcited by 46
  9. 2020R-MADDPG for Partially Observable Environments and Limited Communicationcited by 36
  10. 2020Succinct and Robust Multi-Agent Communication With Temporal Message Controlcited by 29
  11. 2020Multi-Modal Trajectory Prediction of NBA Playerscited by 18
  12. 2019Message-Dropout: An Efficient Training Method for Multi-Agent Deep Reinforcement Learningcited by 17
  13. 2022Model-based Multi-agent Reinforcement Learning: Recent Progress and Prospectscited by 17
  14. 2021MAMBPO: Sample-efficient multi-robot reinforcement learning using learned world modelscited by 16
  15. 2020Information State Embedding in Partially Observable Cooperative Multi-Agent Reinforcement Learningcited by 13
  16. 2020Event-Triggered Multi-agent Reinforcement Learning with Communication under Limited-bandwidth Constraintcited by 10
  17. 2021Time-series Imputation of Temporally-occluded Multiagent Trajectoriescited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2026The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMscited by 10
  2. 2025RecBayes: Recurrent Bayesian Ad Hoc Teamwork in Large Partially Observable Domainscited by 0